I'm trying to install the basic scipy stack (numpy, scipy, matplotlib, pandas, sympy, ipython, nose) into a virtualenv; currently, I'm using OSX Mountain Lion. From the installation pages for these packages, as well as various threads here and around the web, it seems that pandas, sympy, and nose can be installed easily with just pip (although some list numpy/scipy/etc. as dependencies).
However, there seems to be conflicting and kind of convoluted instructions for properly installing numpy, scipy, matplotlib, and, to a certain extent, ipython*. Installing solely with pip doesn't seem to be the proper way to install these packages; it seems that some dependencies need to be installed with homebrew, but various places list different packages to brew install before pip installing numpy/scipy/etc. Is there a comprehensive and current list of package dependencies that need to be installed with homebrew before pip installing numpy, scipy, and matplotlib?
Just as a note, I've looked at tapping homebrew/python for installing numpy, scipy, and matplotlib properly with homebrew, but I want to install into a virtualenv and I don't think I can use homebrew to do that.
Any help would be greatly appreciated; thanks in advance!
*Also, for ipython, the installation page says that pip install ipython[all] should be sufficient, but some other sources (http://www.coderstart.com/setup/python-setup.html; http://www.lowindata.com/2013/installing-scientific-python-on-mac-os-x/) seem to suggest that the qt, pyqt, and zmq packages need to be brew installed first before pip installing ipython; are the homebrew installation necessary or is it fine to just pip install according to the ipython installation page?
Answering my own question here, but hopefully this is helpful; feel free to correct if there are any mistakes. The original version is a gigantic wall of text, so I've added a tl;dr to the top with just the steps to hopefully make the process more clear.
tl;dr: In terminal/bash, go into a virtualenv (if you want to install into one) and enter these commands in order. This is tested for OSX Mountain Lion.
pip install numpy
brew install gcc
pip install scipy
brew install freetype
pip install matplotlib
pip install nose
pip install pandas
pip install sympy
pip install ipython[all]
brew install pyqt
brew install qt
brew install sip
echo "export PYTHONPATH=/usr/local/lib/python2.7/site-packages:$PYTHONPATH" >> ~/.bash_profile
source ~/.bash_profile
*Notes: brew installing pyqt might already install qt and sip; if it does, no need to install qt and sip after installing pyqt. For the second to last line, might be be more reliable to directly c/p that line into ~/.bash_profile, since it might need to be at the top of the ~/.bash_profile contents. Also, brew install pandoc is optional, but necessary for ipython notebook's nbconvert command work properly.
edit 10/13/14 [see edit at bottom]: editing PYTHONPATH in ~/.bash_profile forces virtual environments to inherit global packages; if you want to be able to make isolated environments, don't do the last two steps. Instead, assuming virtualenvwrapper is installed, edit the local postactivate and predeactivate scripts in the bin directory under the virtualenv that contains the scipy stuff.
In postactivate, enter:
export PYTHONPATH=/usr/local/lib/python2.7/site-packages:$PYTHONPATH
In predeactivate, enter:
unset PYTHONPATH
This should edit PYTHONPATH when when the virtualenv with scipy stuff is activated so that ipython qtconsole works, but should reset PYTHONPATH when the virtualenv is deactivated, so that other virtualenvs are not affected by PYTHONPATH changes.
Below is the long narrative version.
Anyway, from some trial and error after I originally posted the question, I found that the following steps work; I used the two sets of instructions that I linked above as general guides, and this was tested on OSX Mountain Lion.
After activating the virtualenv in which the packages are going to be installed, first pip install numpy; this should work as expected and numpy should be installed (note that numpy should be installed first since a lot of the other packages in the scipy stack depend on numpy).
Now, before installing scipy, several sources note that gfortran (this seems to be the most common, but I think any fortran compiler should work) needs to be installed; brew install gfortran returns an error that says that gcc now includes gfortran so the gfortran formula is deprecated. Hence, we brew install gcc (note that even though xcode command line tools, which is generally installed before homebrew, already includes gcc, its version of gcc somehow doesn't work or doesn't include gfortran). After installing gcc, pip install scipy works as expected, and scipy should be installed. Quick aside: brew install gcc installs gcc as well as a bunch of dependencies, namely cloog, gcc, gmp, isl, libmpc, mpfr. [These should all be installed into /usr/local/Cellar, which is the default install location for homebrew.]
For installing matplotlib, freetype needs to be installed first, so we brew install freetype; this should install freetype and libpng, which seems to be a freetype dependency. Afterwards, pip install matplotlib works as expected, installing matplotlib successfully. Note that matplotlib installs with mock (needed to run matplotlib test suite), pyparsing (required for mathtext support), python-dateutil (required for date axis support), six (no reason given) along with it. [These should all be installed, along with any other pip install within a virtualenv, into the site-packages directory within the virtualenv.]
Installing nose, sympy, and pandas simply require pip installing, as they don't have any dependencies that need to be brew installed. However, of these, note that at least pandas depends on numpy, so installing pandas (not sure about the others) after installing numpy is probably preferred. Also, note that pandas installs with pytz (for time zone calculations).
Installing ipython is pretty simple, but setting it up is a little more convoluted. As a heads up, ipython can be used with both a qt console and something called ipython notebook, which have various benefits. You can choose to just install ipython with pip install ipython and install optional dependencies later on as needed, but I installed all the main optional dependencies with pip install ipython[all]. This installs ipython, along with a lot of other packages dependencies (installs with backports.ssl-match-hostname (from tornado), certifi (from tornado), docutils (from sphinx), gnureadline, ipython, jinja2, markupsafe (from jinja2), numpydoc (from ipython[all]), pygments, pyzmq, sphinx, tornado). This should be a good foundation for ipython to use both the standard ipython shell, the qt console, and the ipython notebook. However, it's not fully set up if you want to use either the qt console or the notebook.
To use the qt console, the pyqt, qt, and sip packages must be brew installed, as these are dependencies that cannot be installed with pip; from experience, brew install pyqt seems to install all three packages, but individually installing the three might be a safer bet. After doing this, go into ~/.bash_profile and add the line 'export PYTHONPATH=/usr/local/lib/python2.7/site-packages:$PYTHONPATH' to it; then 'source ~/.bash_profile' in terminal to reload the shell. This should successfully allow the qt console to be launched. [I'm not totally sure why this line needs to be added, since I had already edited the PATH variable to put /usr/local/bin before /usr/bin, but perhaps qt/pyqt/sip were still trying to link themselves with the system default python instead of homebrew installed python.]
The notebook seems to work fine out of the box as far as I've seen, but there's one thing of note: in order to use nbconvert (converting the notebook to different file formats), the pandoc package must be installed, presumably with homebrew. Like qt/pyqt/sip, it can't be installed with pip which is why it wasn't installed with pip install ipython[all].
edit 10/13/14: So apparently, editing PYTHONPATH will nullify empty virtual environments, making global packages also available in a virtualenv (how to isolate virtualenv from local dist-packages?); this, for the most part, defeats the purpose of a virtualenv, assuming that you want to a fresh environment, but is necessary for ipython qtconsole to work correctly.
The fix is to edit (assuming virtualenvwrapper is installed) the local postactivate and predeactivate scripts in the bin file of your virtualenv. In postactivate, enter the line "export PYTHONPATH=/usr/local/lib/python2.7/site-packages:$PYTHONPATH; in predeactivate, enter the line "reset PYTHONPATH". Don't do the last two steps of the original sequence, or delete the line that was added to ~/bash_profile. This should make it so that the changes to PYTHONPATH are made only when the virtualenv with our installed packages is activated, so that qtconsole works, but are reset before the virtualenv deactivates, so that other environments are not affected.
I'm on OSX 10.9, but I had good luck following these instructions and get roughly the same environment that you outlined above
http://hackercodex.com/guide/python-development-environment-on-mac-osx/
Hope this helps
Related
I would like to create some graphics with Python that matplotlib seems to be unable to do well, in terms of drawing closed curves on 3D surfaces. A bit of online research led me to mayavi.mlab for that task (Matplotlib: Plot 3D curve on top of 3D surface). I am currently running Python 3.7.11, but I can upgrade to 3.9 or higher if necessary. I have tried multiple ways to install mayavi and all of them fail in one way or another, including the answers to some of the related questions here. Here are my symptoms:
With Anaconda
I tried both conda install mayavi and conda install -c conda-forge mayavi from the command line. Both of them have the same problem, which is being unable to "Solve the environment" with "frozen solve". Then it tries a flexible solve and still fails. Then it tries a different repo and again is unable to solve the environment with frozen solve.
I also tried Anaconda Navigator for the same thing, but it hangs during a step called "Solving package dependencies" for about six hours, at which point I stopped it.
With pip
Another website told me that pip wheels are more reliable for installing mayavi, but that you should pip install vtk first before mayavi. So I did that: pip install vtk and then pip install mayavi. Both seemed to work during install, in the sense that there were no warnings or errors that I could tell. But as soon as I try to import mayavi in a jupyter notebook I get this error:
This looks to my novice eyes like a problem with a Qt interface of some kind, but beyond that I don't know what is going on.
Summary
I'm obviously not an expert at Python but I don't know what else to try. Usually conda seems to be able to figure out package dependencies and install what is needed but not this time. I would be happy for any advice, and I apologize in advance if I have left out some important information. Is the problem due to using the anaconda3 version of python 3.7 but installing mayavi with pip? I would prefer to install mayavi with anaconda anyway so I would appreciate any pointers about how to do that.
Installing VTK itself (not just python pip package of vtk) is also a prerequisite for Mayavi.
(MacOS only:)
The following steps install both VTK and Mayavi:
Prerequisites:
brew install llvm. which requires manual installation steps in brew info llvm.
pip install wheel # optional. I did it before other steps.
Steps:
brew install vtk # very heavy
pip install vtk
pip install mayavi (not needed, but for some reason I needed it)
brew install qt5
brew info qt5 # slow
# Follow instructions of above. (Set PATH plus LDFLAGS and CPPFLAGS for compiler) and close and open the terminal.
qmake # Test qmake works on terminal (to test the PATH of qt5)
pip install pyqt5 # Slow: builds using clang.
export ETS_TOOLKIT=
export QT_API=pyqt5
brew install pyside # (I installed pyside too, at earlier steps. It might be unnecessary)
I did many other things as well but this is the steps to reproduce the effective steps. I hope I haven't missed something that I have done (or if there are unnecessary steps). Since it changes the configs, it is difficult to revert back and test from scratch.
This installs the combination of following versions: vtk#9.1, qt5.
Also clang 13.1.6 .
Which installs both VTK and Mayavi.
Note, I use venv, so the exact pip command may differ for you if you use conda.
Tested on:
Python: 3.9.13, MacOS: 12.4 Monterey M1
I want to write program in python3 (3.5), hence I installed python3 next to the pre-installed python2 (2.7) on Mac OS X El Captian.
Since my terminal runs python2.7 by default and Numpy is already installed for it, I put alias python=python3 and expected to be able to install Numpy for python3. when I type pip install numpy. This was the generated message:
Requirement already satisfied (use --upgrade to upgrade): numpy in /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages
I also noticed that I have no pip3 even though I am using python3: python --version returned Python 3.5.2, but pip3 install numpy got me -bash: pip3: command not found.
So my questions are:
1) How to install Numpy for python3.x when Numpy is installed on python2.x?
2) How to get pip3?
3) Is it better to use virtual environments, such as Conda, instead of juggling between python2 and python3 on the system?
Thank you from a total n00b
------------------- Update -------------------
Reinstalling python3 also fixed another problem in my case.
When I ran brew doctor, one of the warning message I got was:
Warning: You have unlinked kegs in your Cellar Leaving kegs unlinked can lead to build-trouble and cause brews that depend on those kegs to fail to run properly once built. Run brew link on these: python –
This is a result of me running brew unlink python in order to fix
"Python quit unexpectedly"
when I launch Vim and also
"The ycmd server SHUT DOWN"
Both seem to relate to the YouCompleteMe autocomplete plugin which I downloaded for Python.
I got my idea of removing symlinks from here and here
However, Homebrew evidently did not like the absence of those 39 symlinks.
After uninstall (brew uninstall python3) and then re-install python3 (brew install python3) as Toby suggested, Homebrew gave me
You can install Python packages with
pip3 install <package>
Then when I pip3 install numpy and pip3 install scipy, both executed successfully.
To my surprise, symlinks created during Python installation used to cause aforementioned error messages for Python and YouCompleteMe, but now I open python files using Vim without crash from a fresh Python installation, which definitely created the symlinks.
------------------- Update2 ------------------
After re-installing Anaconda2, the same YouCompleteMe error came back. I suspect Anaconda messed up symlinks.
I would recommend using the Anaconda Python distribution.
The main reasons are as such:
You will have a Python distribution that comes with numpy and the rest of the Scientific Python stack.
The Anaconda Python will be installed under your home directory, with no need for sudo-ing to install other packages.
conda install [put_packagename_here] works alongside pip install [put_packagename_here]; conda install is much 'cleaner' (IMHO, differing opinions are welcome).
If you have a Python 3 environment as your default, then pip works out-of-the-box without needing to remember to do pip3.
conda environments are easier to manage than virtualenv environments, in my opinion. And yes, you can have Python 2 alongside Python 3.
I once messed up my system Python environment - the one that came with my Mac - and it broke iPhoto (back in the day). Since then, I became convinced of needing separate, atomic environments for different projects.
I've detailed more reasons in a personal blog post.
Other distributions, of course, are all good, provided they give you what you need :).
The simplest way on a Mac is with Homebrew:
http://brew.sh/
Install Homebrew, then run:
brew install python3 pip3
Edit --
Python3 includes pip3, but Homebrew occasionally has trouble linking to the correct versions, depending on what has been installed. Running the following command:
brew doctor
And if you see errors relating to python or unlinked kegs, try running:
brew uninstall python python3
And reinstalling after checking brew doctor.
https://unix.stackexchange.com/questions/233519/pip3-linked-to-python-framework-instead-of-homebrew-usr-local-bin
I am managing several modules on an HPC, and want to install some requirements for a tool using pip.
I won't use virtualenv because they don't work well with our module system. I want to install module-local versions of packages and will set PYTHONPATH correctly when the module is loaded, and this has worked just fine when the packages I am installing are not also installed in the default python environment.
What I do not want to do is uninstall the default python's versions of packages while I am installing module-local versions.
For example, one package requires numpy==1.6, and the default version installed with the python I am using is 1.8.0. When I
pip install --install-option="--prefix=$RE_PYTHON" numpy==1.6
where RE_PYTHON points to the top of the module-local site-packages directory, numpy==1.6 installs fine, then pip goes ahead and starts uninstalling 1.8.0 from the tree of the python I am using (why it wants to uninstall a newer version is beyond me but I want to avoid this even when I am doing a local install of e.g. numpy==1.10.1).
How can I prevent pip from doing that? It is really annoying and I have not been able to find a solution that doesn't involve virtualenv.
You have to explicitly tell pip to ignore the current installed package by specifying the -I option (or --ignore-installed). So you should use:
PYTHONUSERBASE=$RE_PYTHON pip install -I --user numpy==1.6
This is mentioned in this answer by Ian Bicking.
So I was looking around at different things to do on Python, like code for flashing text or a timer, but when I copied them into my window, there were constant syntax errors. Now, maybe you're not meant to copy them straight in, but one error I got was 'no module named wx'. I learned that I could get that module by installing wxPython. Problem is, I've tried all 4 options and none of them have worked for me. Which one do I download and how do I set it up using Windows?
Thanks
It's on PyPI. As of wxPython 4, Python 3 is supported.
Unfortunately, PyPI has a package called wx that is stuck at version 3.0.3; be sure to install the package named wxpython instead.
pip install wxpython
Please note that pip will automatically build wxWidgets for you, but it will not install wxWidgets system dependencies such as GTK and OpenGLu. If the above command exits with an error, look above for a message like this:
checking for <something>... not found
checking for <something>... no
configure: error: <prereq> libraries not available
Error running configure
ERROR: failed building widgets
This should give you information about at least one of the packages your system is missing.
The "official" list of prerequisites from the wxWidgets source is:
dpkg-dev
build-essential
libjpeg-dev
libtiff-dev
libsdl1.2-dev
libgstreamer-plugins-base0.10-dev # or 1.0 if available
libnotify-dev
freeglut3
freeglut3-dev
libsm-dev
libgtk-3-dev
libwebkitgtk-3.0-dev # or libwebkit2gtk-4.0-dev if available
libxtst-dev
The actual package names provided by your package manager may not match these exactly, and to be honest, I don't really know the best way to query a package manager to determine what packages provide the libraries you need.
3 steps to install wx-widgets and pygame in python IDLE
Install python 3xxx in your system opting (Add 3xxx to your path).
open python CLI to see whether python is working or not.
then open command prompt (CMD).
type PIP to see whether pip is installed or not.
enter command : pip install wheel
enter command : pip install pygame
To install wxpython
enter command : pip install -U wxPython
Thats all !!
As per home page instruction:
Make sure you have at least version 6.0.8 of pip and 12.0.5 for setuptools.
Install requirements for Linux as outlined in the readme.rst at:
https://github.com/wxWidgets/Phoenix/blob/master/README.rst
Install wxPython-Phoenix (Linux):
sudo pip install --upgrade --trusted-host wxpython.org --pre -f http://wxpython.org/Phoenix/snapshot-builds/ wxPython_Phoenix
Install wxPython-Phoenix (Windows, use the appropriate script folder):
C:\python27\scripts\pip.exe install --upgrade --trusted-host wxpython.org --pre -f http://wxpython.org/Phoenix/snapshot-builds/ wxPython_Phoenix
I installed wxPython as part of the PsychoPy experiment builder dependencies, and had considerable trouble getting it to install properly as well initially. But this was what worked for me at the end. I use Ubuntu 16.04, python 3.5, pip3 19.0.3
pip3 install -U -f https://extras.wxpython.org/wxPython4/extras/linux/gtk3/ubuntu-16.04 wxPython --user
If you use Conda then you may easily setup the environment with wx by one line:
$ conda create -n wxenv python=3 wxPython
Solving environment: done
## Package Plan ##
environment location: /home/user/.conda/envs/wxenv
added / updated specs:
- python=3
- wxpython
The following packages will be downloaded:
package | build
---------------------------|-----------------
[...]
Proceed ([y]/n)?
You need to ensure the versions of your wxPython download matches your installed python language library.
The current downloads wxPython downloads doesn't show any libraries built against python 3. I Believe the python 3 porting project is still ongoing.
If you are not sure of what you are doing I would stick with the 32bit version on windows as there are some Python libraries (ie IIRC, MySQLdb) which don't work with 64 bit python.
So you would then need to download python2.7 for windows x86 and "wxPython3.0-win32-py27 32-bit Python 2.7"
To install wxPython GUI library correctly go to the following page (https://wxpython.org/Phoenix/snapshot-builds/), which contains snapshots builds of wxPython library (Phoenix version) depending on your os and version of Python you want to work.
Then when you downloaded the proper package for your system and python version, simply install it by using pip. In my case I've choosen that one (wxPython_Phoenix-3.0.3.dev2811+ecc4797-cp36-cp36m-win_amd64.whl):
pip install wxPython_Phoenix-3.0.3.dev2811+ecc4797-cp36-cp36m-win_amd64.whl
To check that it has been installed sucessfully on the site-packages folder for your current python environment write:
pip freeze
It's all!
Check the version of wxpython and the version of python you have in your machine.
For python 2.7 use wxPython3.0-win32-3.0.2.0-py27 package
The problem was solved in openSuse simply with
zypper in python-wxWidgets-3_0-devel
Trying pip install before, gave me a lot of trouble (missing traits, missing wx/setup.h, https://github.com/wxWidgets/Phoenix/issues/1644, error: aggregate ‘wxGLAttributes _NullGLAttributes’ has incomplete type and cannot be defined, etc.).
wxpython failed to be installed with pipenv. Pipenv is not able to find wxpython binary so it tries to build wxpython but fails.
CXXFLAGS="-I/opt/homebrew/include" pipenv install wxpython
On my macOS M1 pipenv failed to install wxPython. After a lot of searching I found a forum post which really helped me fix the problem.
Source/Credits: https://forums.wxwidgets.org/viewtopic.php?t=47953&p=203709
Install current development version with:
pip install -U https://github.com/robotframework/RIDE/archive/master.zip
(python < 3.9) Install current Beta version (2.0b1) with:
pip install psutil
pip install -U --pre robotframework-ride
Note that I tried to install wxPython with 'pip install -U wxPython' as per instruction
with no avail. Too many errors to list here. 🤨
I found a solution to the problem!!
I'm working on a 64b machine and Windows 11 operating system using VSCode.
Here is the solution using PowerShell:
Version specs:
pip 22.3.1
virtualenv 20.15.1
python 3.10.8
Create a new virtual environment in the directory where the program resides and
activate. There must be no modules installed.
virtualenv venv
venv/scripts/activate.bat
Install the following in sequence:
pip install pygame
(Not sure why pygame must be installed first, but this was
recommended and it works) 😟
pip install -U wxPython
SUCCESS!!! 🤠
These are the modules installed:
numpy 1.24.1
Pillow 9.4.0
pip 22.3.1
pygame 2.1.2
setuptools 65.4.0
six 1.16.0
wheel 0.37.1
wxPython 4.2.0
VSCode still reports wx as a missing module even when you activate the virtual
environment within. Running the code from the PS command prompt within the virtual
environment is the only working solution.
PS. I am sure there are some conflicts when trying to install wxPython within an
environment where all the other modules are installed.
I'm a Windows user of Python. I used pip to install packages, and I have about a dozen of packages in Python global package repo. However, there were packages I could not install using pip, e.g: numpy, scipy, matplotlib, lxml. To my understanding, installing package by pip requires compiling, and installation failed because they could not be compiled for some reason.
For the workaround, I installed using the installer (.exe) files of the packages. They work all fine, and appear in Control Panel list of installed program as such:
Recently, I can install/update all package smoothly with pip, which I think because I installed Cython and/or have MinGW GNU compiler in PATH. However, seems that pip keeps its own version of installed packages: e.g. pip list still shows matplotlib-1.3.1, numpy-1.7.2 etc.
I test by: pip install -U lxml, lxml in pip list becomes 3.4.1, but the 3.3.3 in the Control Panel is still there. Looks like I'm having 2 version of lxml.
Another test: matplotlib-1.4.2 (latest) was installed by exe installer. But under pip's view it's still 1.3.x. Now calling pip install -U matplotlib, this invokes the latest matplotlib, numpy and some other packages being downloaded, compiled and installed. Only after that, pip list returns the latest version number.
So why this strange behavior? (However import call always result in the later version installed). Should I uninstall all the packages by exe installer and reinstall them by pip for "consistency"?
In the pip way, although installation finished successfully, the compiler threw a bunch of warnings for some packages.