Develop Reference

Site-Packages in Python2.7 without VirtualEnv

When I use pip install matplotlib --upgrade it runs and installs matplotlib and all dependicies, and at the end reads uninstalled version '1.3.1'. Running python and matplotlib.__version__ outputs '1.3.1' again, even though it was just declared uninstalled. How do I switch which python I'm running from, which would hopefully be where pip is installing and updating packages?
Edit: Running Mac OS 10.10.1, using python from bash.
which pip returns /usr/local/bin/pip and which -a python returns
/opt/local/bin/python
/opt/local/bin/python
/Library/Frameworks/Python.framework/Versions/Current/bin/python
/Library/Frameworks/Python.framework/Versions/Current/bin/python
/Library/Frameworks/Python.framework/Versions/Current/bin/python
/Library/Frameworks/Python.framework/Versions/2.6/bin/python
/usr/bin/python
which I'm assuming is the problem. I've deleted 2.6 before, but my Mac uses it and I had to restore my OS. How do I switch amongst them.

Your problem appears to be that you have two or more Python 2.7 installations, and the first python on the PATH comes from one installation (the MacPorts version, at /opt/local/bin/python), while the first pip on the PATH comes from a different one (you haven't said which, but I'd guess it's /usr/local/bin/pip, and it's attached to either the pre-installed Apple Python or yet another Python).
If you want to have lots of Pythons installed all over the filesystem and switch among them without using virtualenv, you just need to write some scripts with modify your PATH. (And your PYTHONPATH, if you use one… but hopefully you don't.)
But I wouldn't recommend that. If you want to use the MacPorts Python:
Make sure you have the MacPorts pip installed (it should be in /opt/local/bin/pip). I don't know if MacPorts has a separate python-pip package, or expects you to install it manually, or otherwise.
If you don't know how to rehash your shell cache, just open a new Terminal window after getting the MacPorts pip installed. Or, to be safe, just explicitly use /opt/local/bin/pip until the next time you logout and back in.
When you write scripts, you may want to #!/opt/local/bin/python instead of #!/usr/bin/env python.
Uninstall every other Python except the MacPorts one and the pre-installed Apple ones.
Any packages you accidentally installed for other Python installations, just reinstall them for MacPorts. Also, you should usually check for ports first, like port install python-ipython or port install python27-ipython, and only fall back to pip if there is no portfile.
As a side note, many people who have this problem also have MacPorts colliding with Homebrew (or sometimes Fink, gentoo-alt, or other package managers, but none of them are very common nowadays). If you're using two such package managers, you really, really shouldn't be unless you know what you're doing. I'd disable one entirely (at least take it off your PATH, if not uninstall it). You will likely have to rebuild a bunch of stuff after you do so, but it's better than having a bunch of stuff that kind of sometimes works…

Simple instructions for installing python easy_install on OSX

I'm a front-end developer with zero server-side experience. I'm trying to learn a little python for an app I'm building, and I literally cannot find anywhere that makes it easy to install easy_install. Everything I read is full of lingo I don't get, like using homebrew or PATHs. And pypi.python.org is TERRIBLY written and designed. So far it's only sent me in circles.
I'm looking for a simple set of instructions in the following format:
Go to x website and click x link
Type x into the terminal and run
Type x to be sure it's installed properly
Please for the benefit of myself and the whole internet, can someone provide this? I'm definitely up for learning how to do some things the hard way, but Python isn't something I need to know very thoroughly right now, and I imagine that's the case for others just looking to get their feet wet.
Assume I have Python installed, but not that I know anything else about using it.
(If there really is a resource out there like this that exists, let me know... I just haven't seen anything like it yet and I've been googling around for about an hour.)
Thank you!

easy_install is part of setuptools. For a while, distribute was a separate fork of setuptools that included its own easy_install, and you had to put a bit of thought into which one you wanted, but that's no longer an issue; the two have merged back into one project. So, the question is, how do you get setuptools?
If you're using Apple's pre-installed versions of Python, you already have setuptools, with a working easy_install, for all versions. You can also use easy_install-2.5, etc., for the earlier versions of Python that Apple also pre-installs. So there's nothing to do.
If you're using a Homebrew Python, it automatically installs setuptools by default, so again, nothing to do.
If you're using a third-party package manager that doesn't automatically install setuptools (as at least was true for MacPorts last time I checked), it probably has its own package for python32-setuptools or python32-easy-install or something like that.
Most of the "extra-batteries" distributions (ActiveState, Enthought, etc.) of Python similarly include setuptools, so again, nothing to do.
If you're using a Python.org binary, a custom-built Python with default configuration, etc., the standard setuptools instructions for "Unix-based Systems including Mac OS X" work fine.
If you installed Python somewhere you don't have write access, you will need to sudo the installation step, as in the second variant.
So:
wget https://bitbucket.org/pypa/setuptools/raw/bootstrap/ez_setup.py -O - | sudo python
And that's all there is to it.
All that being said, you almost never want to use easy_install, except to install two things:
readline, to replace the fake libedit-based support with real libreadline-based support. (And any other packages that don't work with pip, but this is the only really popular one.)
pip, so you never need to use easy_install again.
Most of the cases above that include setuptools also include pip; the one exception is Apple pre-installed Python, where sudo easy_install pip is the simplest way to get pip.
For the cases that don't come with either, the standard documentation for installing pip also works:
Install or upgrade setuptools (as above) if you have a version older than 0.7.
Download get-pip.py.
python get-pip.py
Again, sudo if you need to, and replace the python with python3.3 or python2.6 or /opt/local/mypython/bin/python or whatever to install for any non-default versions.
Finally, for upgrading, once you have any version of pip, and setuptools 0.7 or later, it's as easy as you could imagine:
pip install --upgrade setuptools pip
Again, sudo if necessary, and pip-3.3 or whatever if necessary.
Note that to make easy_install or pip actually useful, you may need a compiler toolchain. The easy way to do this is to install Xcode from the Mac App Store (free as of late 2013), launch Xcode, open Preferences, go to Downloads, and install the "Command Line Tools". This is sufficient for almost everything you'd ever want to do.
The one major exception is SciPy and related packages, for which you will also need a Fortran compiler. See this blog post for Python 2.x and this one for 3.x for details. If all of that looks scary to you, you may be happier using Enthought or one of the other pre-packaged scientific distributions.
One last thing: If you installed a non-Apple Python, and then upgraded your OS X (or Homebrew or MacPorts or Enthought or …) to a newer major version (e.g., from OS X 10.8 to 10.9), you may need some extra steps. For every case but OS X itself, they should be documented on the home page for whichever package manager or Python distribution you're using. For an OS X upgrade, reinstalling your third-party Python is usually the simplest solution if you run into a problem (after first verifying that you actually need a third-party Python in the first place), even if it means you have to reinstall your site packages.

The short correct answer is don't use EasyInstall.
Use pip!

Django - Mac development, environment hell

I was trying to setup Django dev environment on Mac and arrived into a hell. It all started when trying to install PIL, which failed after trying 15 or so different recipes I found on blogs. So I wanted to install the Python, this time 2.7, and reinstall setuptools, easy_install, pip from scratch.
After just installing Python 2.7, and easy_install with setuptools for 2.7, this all in turn created such a mess that is unbelievable. Different version of Python are installed everywhere, easy_install is installed everywhere and points randomly to different python hashbangs (sometimes to #!/usr/bin, #!/usr/local/, #!/Library/...)
Now I can't even do easy_install pip, which I always could. So I'm already in a hell and I haven't even attempted to install MySQL yet.
My question finally is did anyone bump into such problems, it would help enough to know that I'm not alone.
Second, would it be easier to set up the entire environment on Ubuntu than it is on a Mac?
Thirdly, is there any guide that can really clearly explain how to set up but also tear down the stack for Python development on a Mac?

It wouldn't hurt to run a VM with vagrant. This post should tell you more:
http://stevelosh.com/blog/2011/06/django-advice/
Of course using virtualenv should also help alleviate some of these issues.

I've gone through the same hell 2 weeks ago :)
I needed to make working python 2.7 and virtualenv on OSX 10.6.8.
You haven't mentioned virtualenv in your question but I strongly recommend it. That way you minimize amount of globally installed packages. Everything is... cleaner.
My idea is to only have following things globally:
python (from brew)
pip (via easy_install)
virtualenv (via pip)
virtualenvwrapper (via pip)
other through either virtualenv or buildout
I've just checked and pip PIL installs fine within my virtualenv.
Here are notes from this battle (gist.github.com):
#NOTE: .pydistutils.cfg seems to be not compatible with brew install python
#areas I needed to clean before installation
#clean up ~/Library/Python
#clean up .local
brew install python
easy_install pip
pip install virtualenv
pip install virtualenvwrapper
mkdir $HOME/.virtualenvs
Example .bash_profile:
#homebrew
export PATH=/usr/local/bin:/usr/local/sbin:${PATH}
# homebrew python 2.7
export PATH="/usr/local/share/python:${PATH}"
#virtualenv wrapper
export WORKON_HOME=$HOME/.virtualenvs
source /usr/local/share/python/virtualenvwrapper.sh
Good luck!

Second, would it be easier to set up
the entire environment on Ubuntu than
it is on a Mac?
To answer this question (though I never used Mac though): I never had problems setting up a python environment for Django development on Ubuntu. Though in any case you should go with the built-in Python version if possible. Attempting to install any other Python versions usually ends up messy. Luckily with Ubuntu 11.04 the standard version is already 2.7.

I'm using django development environment on a MAC OS X 10.8 with python 2.7. I don't use virtualenv ore some other things.
With all the respect can say that there is NO ANY PROBLEMS to develop on a mac. Mac is a UNIX like system and you've probably seen that all tools for developers have MAC ports.
As for the setup mess. It's a good idea to use virtualenv. As for PIL installation. I needed to compile it with TrueType. As I'm in common with UNIX like environments it was not heavy task for me to compile PIL from sources using GCC (it's already installed on a MAC)... There are some mess with Django to setup virtualenv... There are certainly lots of articles to setup it on Google.
I use Eclipse and write all my PYTHONPATH variables there. You can forget installing everything like in Linux and try not to make anymore mess with installed tools. Try to read THIS article if you feel like you're ok to use Eclipse for your development on a MAC. It also has a recipe to avoid mess with installation of many copies of Python and other dev utils.

Yes I have had problems with MacOS. I think rather than trying to figure it out I just switched to Ubuntu. I use a mac with Ubuntu installed in VMware Fusion. I have developed on both and prefer the Ubuntu because I'm just more comfortable with installing packages and the file structure.
I love using the VM because I'm never scared of having to start over. I can get a whole new OS installed and get the packages with what I use in just a few hours. Not to mention with 6month rollouts I can do complete installs of new versions instead of updates.
Depending on your production environment, it may be beneficial to use an OS that is similar, if you can install a package on ubuntu desktop, you already know how to do it on ubuntu server.

What is the best way to install python 2 on OS X?

A colleague of mine wants to use my python 2 code on his OS X (10.6) machine. My code imports several built-in python packages, including Tkinter and shelve, and also uses third-party packages, including numpy, scipy, matplotlib, and ipython.
I've encountered a few problems with OS X's built-in python. (IDLE doesn't work, for example*). I suspect I should install a more recent version of python, and a different version of Tk.
My questions:
Will having two different versions of python/Tk on the same machine cause problems?
I would like to associate the terminal commands 'python', 'ipython', and 'easy_install' with the more recent version of python. How should I do this?
When I install third-party packages like numpy using a .dmg file, how do I control which version of python numpy installs into?
Is there a better way to do this?
If this process goes well, I'd consider adding OS X instructions to my code's documentation, so I'd like to boil down this process to the simplest, most general approach.
*EDIT: Also, this
EDIT: Thank you everyone for the useful answers. My colleague tried MacPorts, which seems to work well, but has a few speedbumps. First we had to install Xcode from the system install disk. This is not a fast or lightweight install (several GB). Luckily we still had the disk! Once Xcode was installed, MacPorts was easy to install. Python and the python subpackages we needed were also easy to install, but he told me this installation took several hours. Presumably this delay is due to compilation? He had an easy time setting the MacPorts python as default. However, I think we have to change the 'Python Launcher' application by hand, this seems to still default to the system python.
Even though he has a working system now, I'm tempted to ask him to try one of the other solutions. I'm not sure all of my code's potential users will tolerate a multi-hour, multi-gigabyte installation.

I use brew to install all my libraries/compilers/interpreters.
To install python try this:
brew install python
Then add Python's binaries directory to your $PATH in your ~/.profile:
export PATH=`brew --prefix python`/bin:$PATH
I'd recommend you to install pip, virtualenv and virtualenvwrapper to have better control over your environment too.

Have you tried ActivePython?
It includes a package manager (PyPM) that, by default, installs into your home directory (eg: ~/Library/Python/2.7). Main scripts get symlinked in /usr/local/bin; use the included pythonselect to set the active Python version.
You don't have to bother installing .dmg packages, as PyPM is a binary package manager ... therefore you can install non-pure Python packages like NumPy without having to compile things yourself.
ActivePython can use Apple's Tcl/Tk or, if installed, ActiveTcl.
A "simplest, most general approach" in your documentation could be:
Install ActivePython 2.7
Open Terminal and type pypm-2.7 install matplotlib ipython

Using MacPorts, you can install python 2.6, 2.7, 3.1 and 3.2 at the same time, with their own packages, without ever touching the built-in python.
numpy, scipy, matplotlib, and ipython are also available as ports for most of those python versions.
Moreover, if you install the python_select port, you'll be able:
to choose which one of those (plus the built-in python) is the "default" python;
to install python packages through easy_install/pip for the "selected" python, if they're not available as ports.
Add virtualenv to the mix, and you'll have a very, very flexible Python development environment.
As for your questions:
Q1: with MacPorts, no. while not a frequent user, I've installed and used matplotlib in 2.6 and 2.7, switching between the two using python_select.
Q2: easy_install, pip, ipython will be "linked" to the python they were installed by. (but see tip 1)
Q3: it's easier to install one of the py{26,27,xx}-numpy ports, or pip install numpy under your python_select'ed python.
Q4: well, MacPorts is the best thing I know after APT on Debian/Ubuntu... :-)
Now, two tips if you try MacPorts:
MacPorts cleanly installs ports separately from the OS X installation, in an /opt/local directory, and each python version is installed in a /opt/local/Library/Frameworks/Python.framework/Versions/{2.5,2.6,2.7,...} directory. Using python_select cleanly switch the "python" command using links. BUT... the Versions/{2.5,2.6,2.7,...}/bin directory, where python scripts are installed, is not added to the PATH. Just adding: export PATH=/opt/local/Library/Frameworks/Python.framework/Versions/Current/bin:$PATH to your ~/.profile will always give you direct access to the scripts installed for the selected python.
to avoid bad surprises, I've added a echo Selected python is \"$(python_select -s)\" line to my ~/.profile, so I always know which is my currently selected python when opening a session... :-)
Regards,
Georges

In almost all cases, the best python to use is the one from http://python.org/. It sets up the paths correctly and doesn't overwrite anything. DMG package installs usually work automatically, as does python setup.py install, and it's not too hard to get setuptools to work. If you want per-user installs, it is easy to set up .pydistutils.cfg and python automatically recognizes the path install_lib = ~/Library/Python/$py_version_short/site-packages

An addendum regarding the usage of brew:
Since some time, brew install python will install python3.
If you intend to install python2, you want to use
brew install python#2
It is perfectly fine to install both python and python3 using brew!

Here is an old post that answers your questions too.
In general it is not a problem at all to have more than one python installation on your machine. You just have to watch out which one you are calling on the command line.
>> which python
... helps to identify where your python binary is located. The original Mac OS X python is usually at "/usr/bin/python"
I personally use the MacPorts python installation. It also supports you with the installation of modules. (see link above)

I have 4 versions of python on my MacBook Pro. 2 from the original install of OS X 10.6 and a subsequent update, then self installed copies of python 2.7 and 3.2. You can update the python command to point at any of the versions. They all install in separate directories and cause no problems with each other.
I'm not sure what will happen when you install from a .dmg file. I believe it will simply use whatever version python points to.
This post on superuser.com answers your questions on changing default paths.

What is the most compatible way to install python modules on a Mac?

I'm starting to learn python and loving it. I work on a Mac mainly as well as Linux. I'm finding that on Linux (Ubuntu 9.04 mostly) when I install a python module using apt-get it works fine. I can import it with no trouble.
On the Mac, I'm used to using Macports to install all the Unixy stuff. However, I'm finding that most of the python modules I install with it are not being seen by python. I've spent some time playing around with PATH settings and using python_select . Nothing has really worked and at this point I'm not really understanding, instead I'm just poking around.
I get the impression that Macports isn't universally loved for managing python modules. I'd like to start fresh using a more "accepted" (if that's the right word) approach.
So, I was wondering, what is the method that Mac python developers use to manage their modules?
Bonus questions:
Do you use Apple's python, or some other version?
Do you compile everything from source or is there a package manger that works well (Fink?).

The most popular way to manage python packages (if you're not using your system package manager) is to use setuptools and easy_install. It is probably already installed on your system. Use it like this:
easy_install django
easy_install uses the Python Package Index which is an amazing resource for python developers. Have a look around to see what packages are available.
A better option is pip, which is gaining traction, as it attempts to fix a lot of the problems associated with easy_install. Pip uses the same package repository as easy_install, it just works better. Really the only time use need to use easy_install is for this command:
easy_install pip
After that, use:
pip install django
At some point you will probably want to learn a bit about virtualenv. If you do a lot of python development on projects with conflicting package requirements, virtualenv is a godsend. It will allow you to have completely different versions of various packages, and switch between them easily depending your needs.
Regarding which python to use, sticking with Apple's python will give you the least headaches, but If you need a newer version (Leopard is 2.5.1 I believe), I would go with the macports python 2.6.

Your question is already three years old and there are some details not covered in other answers:
Most people I know use HomeBrew or MacPorts, I prefer MacPorts because of its clean cut of what is a default Mac OS X environment and my development setup. Just move out your /opt folder and test your packages with a normal user Python environment
MacPorts is only portable within Mac, but with easy_install or pip you will learn how to setup your environment in any platform (Win/Mac/Linux/Bsd...). Furthermore it will always be more up to date and with more packages
I personally let MacPorts handle my Python modules to keep everything updated. Like any other high level package manager (ie: apt-get) it is much better for the heavy lifting of modules with lots of binary dependencies. There is no way I would build my Qt bindings (PySide) with easy_install or pip. Qt is huge and takes a lot to compile. As soon as you want a Python package that needs a library used by non Python programs, try to avoid easy_install or pip
At some point you will find that there are some packages missing within MacPorts. I do not believe that MacPorts will ever give you the whole CheeseShop. For example, recently I needed the Elixir module, but MacPorts only offers py25-elixir and py26-elixir, no py27 version. In cases like these you have:
pip-2.7 install --user elixir
( make sure you always type pip-(version) )
That will build an extra Python library in your home dir. Yes, Python will work with more than one library location: one controlled by MacPorts and a user local one for everything missing within MacPorts.
Now notice that I favor pip over easy_install. There is a good reason you should avoid setuptools and easy_install. Here is a good explanation and I try to keep away from them. One very useful feature of pip is giving you a list of all the modules (along their versions) that you installed with MacPorts, easy_install and pip itself:
pip-2.7 freeze
If you already started using easy_install, don't worry, pip can recognize everything done already by easy_install and even upgrade the packages installed with it.
If you are a developer keep an eye on virtualenv for controlling different setups and combinations of module versions. Other answers mention it already, what is not mentioned so far is the Tox module, a tool for testing that your package installs correctly with different Python versions.
Although I usually do not have version conflicts, I like to have virtualenv to set up a clean environment and get a clear view of my packages dependencies. That way I never forget any dependencies in my setup.py
If you go for MacPorts be aware that multiple versions of the same package are not selected anymore like the old Debian style with an extra python_select package (it is still there for compatibility). Now you have the select command to choose which Python version will be used (you can even select the Apple installed ones):
$ port select python
Available versions for python:
none
python25-apple
python26-apple
python27 (active)
python27-apple
python32
$ port select python python32
Add tox on top of it and your programs should be really portable

Please see Python OS X development environment. The best way is to use MacPorts. Download and install MacPorts, then install Python via MacPorts by typing the following commands in the Terminal:
sudo port install python26 python_select
sudo port select --set python python26
OR
sudo port install python30 python_select
sudo port select --set python python30
Use the first set of commands to install Python 2.6 and the second set to install Python 3.0. Then use:
sudo port install py26-packagename
OR
sudo port install py30-packagename
In the above commands, replace packagename with the name of the package, for example:
sudo port install py26-setuptools
These commands will automatically install the package (and its dependencies) for the given Python version.
For a full list of available packages for Python, type:
port list | grep py26-
OR
port list | grep py30-
Which command you use depends on which version of Python you chose to install.

I use MacPorts to install Python and any third-party modules tracked by MacPorts into /opt/local, and I install any manually installed modules (those not in the MacPorts repository) into /usr/local, and this has never caused any problems. I think you may be confused as to the use of certain MacPorts scripts and environment variables.
MacPorts python_select is used to select the "current" version of Python, but it has nothing to do with modules. This allows you to, e.g., install both Python 2.5 and Python 2.6 using MacPorts, and switch between installs.
The $PATH environment variables does not affect what Python modules are loaded. $PYTHONPATH is what you are looking for. $PYTHONPATH should point to directories containing Python modules you want to load. In my case, my $PYTHONPATH variable contains /usr/local/lib/python26/site-packages. If you use MacPorts' Python, it sets up the other proper directories for you, so you only need to add additional paths to $PYTHONPATH. But again, $PATH isn't used at all when Python searches for modules you have installed.
$PATH is used to find executables, so if you install MacPorts' Python, make sure /opt/local/bin is in your $PATH.

There's nothing wrong with using a MacPorts Python installation. If you are installing python modules from MacPorts but then not seeing them, that likely means you are not invoking the MacPorts python you installed to. In a terminal shell, you can use absolute paths to invoke the various Pythons that may be installed. For example:
$ /usr/bin/python2.5 # Apple-supplied 2.5 (Leopard)
$ /opt/local/bin/python2.5 # MacPorts 2.5
$ /opt/local/bin/python2.6 # MacPorts 2.6
$ /usr/local/bin/python2.6 # python.org (MacPython) 2.6
$ /usr/local/bin/python3.1 # python.org (MacPython) 3.1
To get the right python by default requires ensuring your shell $PATH is set properly to ensure that the right executable is found first. Another solution is to define shell aliases to the various pythons.
A python.org (MacPython) installation is fine, too, as others have suggested. easy_install can help but, again, because each Python instance may have its own easy_install command, make sure you are invoking the right easy_install.

If you use Python from MacPorts, it has it's own easy_install located at: /opt/local/bin/easy_install-2.6 (for py26, that is). It's not the same one as simply calling easy_install directly, even if you used python_select to change your default python command.

Have you looked into easy_install at all? It won't synchronize your macports or anything like that, but it will automatically download the latest package and all necessary dependencies, i.e.
easy_install nose
for the nose unit testing package, or
easy_install trac
for the trac bug tracker.
There's a bit more information on their EasyInstall page too.

For MacPython installations, I found an effective solution to fixing the problem with setuptools (easy_install) in this blog post:
http://droidism.com/getting-running-with-django-and-macpython-26-on-leopard
One handy tip includes finding out which version of python is active in the terminal:
which python

When you install modules with MacPorts, it does not go into Apple's version of Python. Instead those modules are installed onto the MacPorts version of Python selected.
You can change which version of Python is used by default using a mac port called python_select. instructions here.
Also, there's easy_install. Which will use python to install python modules.

You may already have pip3 pre-installed, so just try it!

Regarding which python version to use, Mac OS usually ships an old version of python. It's a good idea to upgrade to a newer version. You can download a .dmg from http://www.python.org/download/ . If you do that, remember to update the path. You can find the exact commands here http://farmdev.com/thoughts/66/python-3-0-on-mac-os-x-alongside-2-6-2-5-etc-/

I use easy_install with Apple's Python, and it works like a charm.

Directly install one of the fink packages (Django 1.6 as of 2013-Nov)
fink install django-py27
fink install django-py33
Or create yourself a virtualenv:
fink install virtualenv-py27
virtualenv django-env
source django-env/bin/activate
pip install django
deactivate # when you are done
Or use fink django plus any other pip installed packages in a virtualenv
fink install django-py27
fink install virtualenv-py27
virtualenv django-env --system-site-packages
source django-env/bin/activate
# django already installed
pip install django-analytical # or anything else you might want
deactivate # back to your normally scheduled programming