Develop Reference

Trying to install new libraries on Spyder (Python 3.7)

I am new to python and I am having trouble downloading some new libraries that don't appear to be pre-installed. The names of them are rasterio and retrying. I continuously get an invalid syntax error, and I have tried using some different suggestions found online.
pip install rasterio
Above is the way I was attempting to do it, and I also tried without the pip in there, but neither option worked. I am new enough to the language that troubleshooting is very difficult. Any help is greatly appreciated!

There are several ways to install rasterio, using anaconda you have the documentation here: https://anaconda.org/conda-forge/rasterio
If you are on windows installing rasterio using pip install can be difficult since it requires binary wheels, but here is the tutorial: https://rasterio.readthedocs.io/en/latest/installation.html
I advise you to use linux since it is much easier to install packages, you can either install a dual boot or a virtual box.

I suppose you are using Spyder IDE with Anaconda. So, to install rasterio package you can open anaconda prompt and type:
conda install -c conda-forge rasterio
For further information about package installation check out this link.
Also, to install any conda packages just google it once, you will find instructions to install it mostly in anaconda's official site.
Edit:
Please remove the rasterio package once. And install it again, see if works. If it doesn't work create a new environment like: conda create --name myenv. Then install rasterio again.
If this still doesn't work, try to install rasterio from here like conda install -c ioos rasterio.

Got problems in installing GEOS for basemap in windows 10

I know that this problem has been asked before, and hope that the administrators will not regard this question as a duplicate for it is a bit different. I am using Python 3.6, Windows 10 system with 64 bit. I've successfully installed pip and have used pip to install a bunch of packages (including numpy, matplotlib, mpl_toolkits and so on). But when I install the basemap and run the from mpl_toolkits.basemap install Basemap, an error occurs saying that there is no module named "mpl_toolkits.basemap.
I go to the document of basemap and it says that GEOS of 3.1.1 is required. And when I run the pip install geos, what I get is only geos-0.2.1.
https://i.stack.imgur.com/cuv9Q.jpg
(so inconvenient to upload pictures)
Is their any alternatives to download GEOS for the basemap installation? I know there's a way for Linux to install, but it does not seem to work on Windows.

I encountered similar problem and finally managed installing basemap via Anaconda in my windows7 environment. I have tried following basemap official installation instruction in Unix Shell and pip install xxx.whl methods, and both failed. I think the possible reasons can be some file not compatible with Windows or not up to date.
Basemap installation via Anaconda:
1. Install Anaconda following official instruction.
2. Install basemap using Anaconda Prompt (be sure to use Anaconda Prompt), run conda install -c anaconda-forge basemap. This may prompt anaconda upgrade, accept it. Then anaconda will install the package and the prerequisites, including latest geos, for you automatically.
3. Run with your favorite IDE. I run jupyter notebook from ANACONDA PROMPT, this is important. Since I have two versions of python, and basemap can not be loaded if I initiate jupyter notebook from cmd.
Have to mention that I did install PROJ4 in advance, so I am not sure if Anaconda will install it for you or not if you don't have one. Basemap official installation instruction: https://matplotlib.org/basemap/users/installing.html, you can also find PROJ4 instruction linkage there.
Hope this could help.

How to install SciPy stack with pip and homebrew?

I'm trying to install the basic scipy stack (numpy, scipy, matplotlib, pandas, sympy, ipython, nose) into a virtualenv; currently, I'm using OSX Mountain Lion. From the installation pages for these packages, as well as various threads here and around the web, it seems that pandas, sympy, and nose can be installed easily with just pip (although some list numpy/scipy/etc. as dependencies).
However, there seems to be conflicting and kind of convoluted instructions for properly installing numpy, scipy, matplotlib, and, to a certain extent, ipython*. Installing solely with pip doesn't seem to be the proper way to install these packages; it seems that some dependencies need to be installed with homebrew, but various places list different packages to brew install before pip installing numpy/scipy/etc. Is there a comprehensive and current list of package dependencies that need to be installed with homebrew before pip installing numpy, scipy, and matplotlib?
Just as a note, I've looked at tapping homebrew/python for installing numpy, scipy, and matplotlib properly with homebrew, but I want to install into a virtualenv and I don't think I can use homebrew to do that.
Any help would be greatly appreciated; thanks in advance!
*Also, for ipython, the installation page says that pip install ipython[all] should be sufficient, but some other sources (http://www.coderstart.com/setup/python-setup.html; http://www.lowindata.com/2013/installing-scientific-python-on-mac-os-x/) seem to suggest that the qt, pyqt, and zmq packages need to be brew installed first before pip installing ipython; are the homebrew installation necessary or is it fine to just pip install according to the ipython installation page?

Answering my own question here, but hopefully this is helpful; feel free to correct if there are any mistakes. The original version is a gigantic wall of text, so I've added a tl;dr to the top with just the steps to hopefully make the process more clear.
tl;dr: In terminal/bash, go into a virtualenv (if you want to install into one) and enter these commands in order. This is tested for OSX Mountain Lion.
pip install numpy
brew install gcc
pip install scipy
brew install freetype
pip install matplotlib
pip install nose
pip install pandas
pip install sympy
pip install ipython[all]
brew install pyqt
brew install qt
brew install sip
echo "export PYTHONPATH=/usr/local/lib/python2.7/site-packages:$PYTHONPATH" >> ~/.bash_profile
source ~/.bash_profile
*Notes: brew installing pyqt might already install qt and sip; if it does, no need to install qt and sip after installing pyqt. For the second to last line, might be be more reliable to directly c/p that line into ~/.bash_profile, since it might need to be at the top of the ~/.bash_profile contents. Also, brew install pandoc is optional, but necessary for ipython notebook's nbconvert command work properly.
edit 10/13/14 [see edit at bottom]: editing PYTHONPATH in ~/.bash_profile forces virtual environments to inherit global packages; if you want to be able to make isolated environments, don't do the last two steps. Instead, assuming virtualenvwrapper is installed, edit the local postactivate and predeactivate scripts in the bin directory under the virtualenv that contains the scipy stuff.
In postactivate, enter:
export PYTHONPATH=/usr/local/lib/python2.7/site-packages:$PYTHONPATH
In predeactivate, enter:
unset PYTHONPATH
This should edit PYTHONPATH when when the virtualenv with scipy stuff is activated so that ipython qtconsole works, but should reset PYTHONPATH when the virtualenv is deactivated, so that other virtualenvs are not affected by PYTHONPATH changes.
Below is the long narrative version.
Anyway, from some trial and error after I originally posted the question, I found that the following steps work; I used the two sets of instructions that I linked above as general guides, and this was tested on OSX Mountain Lion.
After activating the virtualenv in which the packages are going to be installed, first pip install numpy; this should work as expected and numpy should be installed (note that numpy should be installed first since a lot of the other packages in the scipy stack depend on numpy).
Now, before installing scipy, several sources note that gfortran (this seems to be the most common, but I think any fortran compiler should work) needs to be installed; brew install gfortran returns an error that says that gcc now includes gfortran so the gfortran formula is deprecated. Hence, we brew install gcc (note that even though xcode command line tools, which is generally installed before homebrew, already includes gcc, its version of gcc somehow doesn't work or doesn't include gfortran). After installing gcc, pip install scipy works as expected, and scipy should be installed. Quick aside: brew install gcc installs gcc as well as a bunch of dependencies, namely cloog, gcc, gmp, isl, libmpc, mpfr. [These should all be installed into /usr/local/Cellar, which is the default install location for homebrew.]
For installing matplotlib, freetype needs to be installed first, so we brew install freetype; this should install freetype and libpng, which seems to be a freetype dependency. Afterwards, pip install matplotlib works as expected, installing matplotlib successfully. Note that matplotlib installs with mock (needed to run matplotlib test suite), pyparsing (required for mathtext support), python-dateutil (required for date axis support), six (no reason given) along with it. [These should all be installed, along with any other pip install within a virtualenv, into the site-packages directory within the virtualenv.]
Installing nose, sympy, and pandas simply require pip installing, as they don't have any dependencies that need to be brew installed. However, of these, note that at least pandas depends on numpy, so installing pandas (not sure about the others) after installing numpy is probably preferred. Also, note that pandas installs with pytz (for time zone calculations).
Installing ipython is pretty simple, but setting it up is a little more convoluted. As a heads up, ipython can be used with both a qt console and something called ipython notebook, which have various benefits. You can choose to just install ipython with pip install ipython and install optional dependencies later on as needed, but I installed all the main optional dependencies with pip install ipython[all]. This installs ipython, along with a lot of other packages dependencies (installs with backports.ssl-match-hostname (from tornado), certifi (from tornado), docutils (from sphinx), gnureadline, ipython, jinja2, markupsafe (from jinja2), numpydoc (from ipython[all]), pygments, pyzmq, sphinx, tornado). This should be a good foundation for ipython to use both the standard ipython shell, the qt console, and the ipython notebook. However, it's not fully set up if you want to use either the qt console or the notebook.
To use the qt console, the pyqt, qt, and sip packages must be brew installed, as these are dependencies that cannot be installed with pip; from experience, brew install pyqt seems to install all three packages, but individually installing the three might be a safer bet. After doing this, go into ~/.bash_profile and add the line 'export PYTHONPATH=/usr/local/lib/python2.7/site-packages:$PYTHONPATH' to it; then 'source ~/.bash_profile' in terminal to reload the shell. This should successfully allow the qt console to be launched. [I'm not totally sure why this line needs to be added, since I had already edited the PATH variable to put /usr/local/bin before /usr/bin, but perhaps qt/pyqt/sip were still trying to link themselves with the system default python instead of homebrew installed python.]
The notebook seems to work fine out of the box as far as I've seen, but there's one thing of note: in order to use nbconvert (converting the notebook to different file formats), the pandoc package must be installed, presumably with homebrew. Like qt/pyqt/sip, it can't be installed with pip which is why it wasn't installed with pip install ipython[all].
edit 10/13/14: So apparently, editing PYTHONPATH will nullify empty virtual environments, making global packages also available in a virtualenv (how to isolate virtualenv from local dist-packages?); this, for the most part, defeats the purpose of a virtualenv, assuming that you want to a fresh environment, but is necessary for ipython qtconsole to work correctly.
The fix is to edit (assuming virtualenvwrapper is installed) the local postactivate and predeactivate scripts in the bin file of your virtualenv. In postactivate, enter the line "export PYTHONPATH=/usr/local/lib/python2.7/site-packages:$PYTHONPATH; in predeactivate, enter the line "reset PYTHONPATH". Don't do the last two steps of the original sequence, or delete the line that was added to ~/bash_profile. This should make it so that the changes to PYTHONPATH are made only when the virtualenv with our installed packages is activated, so that qtconsole works, but are reset before the virtualenv deactivates, so that other environments are not affected.

I'm on OSX 10.9, but I had good luck following these instructions and get roughly the same environment that you outlined above
http://hackercodex.com/guide/python-development-environment-on-mac-osx/
Hope this helps

pip install matplotlib fails on mavericks with homebrew

I am trying to install the Scipy python stack, but having issues.
I have a new mac os with mavericks (10.9).
I have installed homebrew and am using brew to manage installations.
I used
% brew install python
then (I verified pip is from my homebrew installation)
% pip install numpy [this works]
% pip install scipy [this works]
% pip install matplotlib OR pip install http://downloads.sourceforge.net/project/matplotlib/matplotlib/matplotlib-1.3.1/matplotlib-1.3.1.tar.gz
Both fail with the same error. After checking the backend, I get
Fatal Python error: PyThreadState_Get: no current thread
The error message from mac os X (the pop up error window) shows errors very similar to those in the previous thread
Homebrew + Python on mac os x 10.8: Fatal Python error: PyThreadState_Get: no current thread importing mapnik
It seems I need to fix the python bindings, as some of them linked against the /System/Library python that ships with the mac. What is not clear to me from the above post is
which of the Binary Images need to be "re-linked"? all of them? some seem quite general an not related to matplotlib
how do I actually use install_name_tool to fix the link? As I mentioned, some seem quite general - do I really want to change the path? Do I cp the dylib from old to new?
I have also tried
brew uninstall python
and re-brewing, all to get the same error. I am trying not to use EPD because EPD installs its own libxml2 and others, which I need for other software I will install, and this has caused me problems on an old machine. I am open to considering a virtualenv, but was hoping to learn to just fix my current problem if possible.
Thanks,
Andre
PS, when pip install matplotlib`, the following warning appears, which seems odd because matplotlib is such a staple piece of software
You are installing an externally hosted file. Future versions of pip will default to disallowing externally hosted files.
You are installing a potentially insecure and unverifiable file. Future versions of pip will default to disallowing insecure files.

This is an issue in pygtk. You can test it with brew (it should throw the same error):
brew test pygtk
I ran into the same problem, and got past that error with the following steps from https://github.com/mxcl/homebrew/issues/13654
brew rm py2cairo
brew install py2cairo
You can test that pygtk is working using brew test pygtk again.
I'm still having other problems getting pip install matplotlib to work, so you may also run into more problems, but that should get you past that particular issue.

I had the same problem, and after some experimentation it seems that installing under Python 3 works more reliably. i.e. this installed without error:
pip3 install matplotlib
I then had to re-install everything else using pip3, and make sure I was running under python3 at execution time.

brew install freetype
it works on mac os sierra!

MayaVi installation on Qt4 leads to segmentation faults

I am trying to install MayaVi on my computer. I'm on a MacBook Air with OS X 10.6.8, 4 GB RAM.
My python and most of my stack is built through Homebrew or pip. In particular this is true for python2.7, ipython, Qt4, numpy, scipy, vtk, etc. VTK was installed with --python and --qt-extern flags in Homebrew, and all the builds were fine. I do not have wxPython installed, as I failed to get it working through pip, manual building, or the binaries.
If I run ipython -q4thread, I cannot successfully execute from mayavi import mlab as I get an error related to usage of two different APIs for PyQt: http://groups.google.com/group/spyderlib/browse_thread/thread/36a35baec74ca144
However, if I run ipython alone, I can successfully run from mayavi import mlab. Then I try to follow this example: http://github.enthought.com/mayavi/mayavi/example_using_with_scipy.html, which includes these commands:
import numpy as np
def V(x, y, z):
""" A 3D sinusoidal lattice with a parabolic confinement. """
return np.cos(10*x) + np.cos(10*y) + np.cos(10*z) + 2*(x**2 + y**2 + z**2)
X, Y, Z = np.mgrid[-2:2:100j, -2:2:100j, -2:2:100j]
from mayavi import mlab
mlab.contour3d(X, Y, Z, V)
This all works fine, and I get a window that pops up and I can rotate the 3d plot etc. However, then I click on the icon to open up the pipeline, which opens another window. The tutorial then says to double-click on the "isosurface" to change its properties. As soon as I do that, I get a reproducible segmentation fault.
I have no idea how to begin to figure out what the problem is.
Alternatively, I can run MayaVi from the command line: mayavi2, and the GUI pops up. When I do so I get the following output:
Warning: Unable to import the wx backend for pyface due to traceback: Traceback (most recent call last):
File "/usr/local/Cellar/python/2.7.1/lib/python2.7/site-packages/pyface/toolkit.py", line 45, in _init_toolkit
be = import_toolkit(tk)
File "/usr/local/Cellar/python/2.7.1/lib/python2.7/site-packages/pyface/toolkit.py", line 31, in import_toolkit
__import__(be + 'init')
File "/usr/local/Cellar/python/2.7.1/lib/python2.7/site-packages/pyface/ui/wx/init.py", line 14, in <module>
import wx
ImportError: No module named wx
Although the GUI does pop up, if I try to do any operation, I get a segfault.
Please let me know if you have any thoughts on troubleshooting this, or perhaps guidance on reinstalling the package successfully.
Thanks!
Uri

I also have a MacBook Air with OS X 10.7.4 (Lion). I spent a lot of time back in February to get mayavi working, eventually succeeding to a point where I could produce scientific plots. That still means that some of the examples and tests wouldn't run correctly, and the GUI produced lots of error messages in the console, but didn't crash (most of the time). Now, I managed to break my Python installation (probably by upgrading numpy), so I needed to reinstall. Alas, it didn't get any easier 6 months later!
There is still no viable precompiled option. I tried the Enthought Python Distribution from here: http://www.enthought.com/repo/.epd_academic_installers. Still no 64 bit version with ETS for Mac, components are ancient, mayavi based on wxPython looks ugly and feels slow and unresponsive. So back to brew and pip. After a lot of trial and error, here is what worked more or less:
1. Prerequisites
Hide/uninstall the broken Homebrew Python installation:
mv /usr/local/lib/python2.7/site-packages /usr/local/lib/python2.7/site-packages-old
mv /usr/local/share/python /usr/local/share/python-old
brew uninstall python pyqt pyside vtk
From Xcode (4.4.1), choose llvm-gcc as the compiler (I had less success with clang):
cd /usr/bin
sudo rm cc c++
sudo ln -s gcc cc
sudo ln -s g++ c++
2. Installation
2.1 Python
brew install python --framework --universal
Point to the new installation:
cd /System/Library/Frameworks/Python.framework/Versions
rm Current
ln -s /usr/local/Cellar/python/2.7.3/Frameworks/Python.framework/Versions/Current .
If the EPD is installed, one needs to do the same in /Library/Frameworks/Python.framework/Versions, otherwise the EPD will take priority.
2.2 Numpy
Install numpy from source (pip 1.2 currently doesn't install npymath.ini correctly, which will make scipy fail to build):
git clone https://github.com/numpy/numpy.git
cd numpy
git checkout v1.6.2
python setup.py install
Since so many packages link against numpy, I really recommend checking out an official release, 1.6.2 as of today. Next make the numpy headers visible:
cd /usr/local/Cellar/python/2.7.3/Frameworks/Python.framework/Versions/2.7/include/python2.7
ln -s /usr/local/lib/python2.7/site-packages/numpy/core/include/numpy .
2.3 qt, pyqt, vtk
Install qt (4.8.2) and pyqt (4.9.4):
brew install qt
brew install pyqt
One can also install pyside (1.1.1)
brew install pyside
but this appears to produce more crashes with mayavi. In any case, append
export QT_API=pyqt
to .bashrc to avoid any mixups. Next install vtk (5.10.0; 5.8.0 works just as well)
brew install vtk --python --tcl --examples --qt
...and wait (compilation takes around 40 minutes). The --qt flag isn't strictly necessary, but it doesn't harm either.
Next, download vtk data from http://www.vtk.org/files/release/5.10/vtkdata-5.10.0.tar.gz, unpack somewhere and have the VTK_DATA_ROOT environment variable point to the VTKData directory. Run some examples in
/usr/local/share/vtk/Examples
Launch python examples with python <example>.py and TCL examples with vtk <example.tcl>. Works absolutely beautifully, right? So let's see how mayavi messes it all up next...
2.4 ETS and mayavi
Although
pip install mayavi
gets you somewhere, it appears to produce more crashes than installing the bleeding edge from github. Get https://github.com/enthought/ets/raw/master/ets.py and run
python ets.py clone
to clone the ETS git repository. You only need apptools, mayavi, pyface, traits and traitsui, possibly envisage if you want to run mayavi2 from the command line. So delete the other directories and do
python ets.py develop
This should allow you to run some of the examples in the repository in mayavi/examples/tvtk and mayavi/examples/mayavi by running python <example>.py and play around with the GUI. There are lots of console errors and lots of GUI features that don't work, like selecting a LUT table. But it hopefully doesn't crash.
2.5 scipy, matplotlib, ipython
I prefer to get the bleeding edge from github for scipy and matplotlib. First install all dependencies using brew. Then
git clone https://github.com/scipy/scipy.git
cd scipy
python setup.py install
git clone https://github.com/matplotlib/matplotlib.git
cd matplotlib
python setup.py install
Then install ipython (0.13) using pip. For the qtconsole install first:
pip install pygments
pip install pyzmq
then
pip install ipython
There are a few possible ways to launch ipython for use with mayavi, depending on who controls the QT event loop:
ipython without mlab.show(): Hangs.
ipython with mlab.show(): Works, but prompt may become unresponsive.
ipython --gui=qt or ipython --pylab=qt: Works, but "Save" dialog closes immediately.
ipython qtconsole without mlab.show(): Hangs.
ipython qtconsole with mlab.show(): Works.
ipython qtconsole --gui=qt or ipython qtconsole --pylab=qt: Works.
Good luck!