Develop Reference

Where did python3.4 download to on my mac?

I decided today I better download python 3.4. So I go to the python/downloads page and do that. Now I am trying to make a new virtualenv using my new python module, mkvirtualenv -p python3.4 sandbox, but I get an error that it can't find my python executable.
The executable /Users/croberts/python3.4 (from --python=/Users/croberts/python3.4) does not exist
This is understandable, but I can't figure out where it is. The old versions of python are in /usr/bin/ but the new one didn't get installed there. How do you search for where a program is using the terminal?

According to the ReadMe.txt bundled with the installer, Python installs default to /Library/frameworks/Python.framework. I just did a test install and it ended up here specifically:
/Library/frameworks/Python.framework/versions/3.4/Python
Unless you changed the settings on the installer, that's where it should live. (If you installed it from source, I'd assume you'd know where you put it.) Something to note is that if you use Homebrew (and possibly other OSX package managers like MacPorts, though I haven't used it), installing Python through the PPC installer will cause a warning:
Warning: Python is installed at /Library/Frameworks/Python.framework
Homebrew only supports building against the System-provided Python or a
brewed Python. In particular, Pythons installed to /Library can interfere
with other software installs.
So if you do use Homebrew (and it's great), it's best to simply use brew install python3, which will put it into /usr/local/bin. Then you can alias python (or python3) to that version, instead.

How to install libxml2 for python when using pyenv

Here are the details of my python install:
OS X 10.9
homebrew used to install pyenv (https://github.com/yyuu/pyenv)
python was installed via pyenv install 2.7.6
custom virtualenv just for this project
Given the above python install, how would someone set up libxml2?
So far, I've tried brew install libxml2, hacking the recipe to have --with-python. That seems to compile, but then python starts having SegFaults when I try to use libxml2.
Thanks in advance!

It may possibly be related to packages being compiled against old libxml2. You can check libxml2 origin with gdb, run gdb python and execute command that lead to SegFault.
One option why it is happening is that brew installed libxml2 to local folder, say /usr/local, and your packages are compiled against libraries shipped with OS (usually old and buggy on OSX, not sure for Mavericks) from /usr/lib are being used.

Found the solution.
brew install libxml2 --with-python
Some files were installed into /usr/local/lib/python2.7/site-packages/libxml2*. I copied them to my virtualenv's site-packages and voila things worked!
Symlinking might be something better to try.

How to handle different versions of python protobuf

My python package contains a lot of files compiled by python-protobuf (python2-protobuf-2.5.0 on Arch Linux), I installed the package on Ubuntu server 12.04.3 (which have python-protobuf-2.4.1), tried to run the code, and hit the following error:
from google.protobuf.internal import enum_type_wrapper
ImportError: cannot import name enum_type_wrapper
I think it's because the protobuf modules in my package are compiled by protobuf-2.5.0 and they do not work with protobuf-2.4.1.
I have no idea of the environments in which my code may run, the version of protobuf may vary. How to make my package work with both protobuf 2.4 and 2.5?
(A possible way: include two different sets of protobuf libraries (one compiled by 2.4.1, the other compiled by 2.5.0) in my package, get google.protobuf version at runtime and select the protobuf libraries to import. Is it possible?

You need to specify the version of protobuf that will work with in your setup.py in the list install_requires=['protobuf>=2.5.0']. With a Python package, you can put just the name or the exact versions that will run with the package using ==. I believe you can also specify != for specific versions.
If you are not packaging it with a setup.py, you should set up a virtualenv and put a file install_requires.txt with all the specific python packages and versions in the root of the project.
That might look like:
$ cd ../project
$ virtualenv project_venv
$ source project_venv/bin/activate
$ cd project
$ pip install protobuf>=2.5.0
$ pip freeze > ./requirements.txt
Then someone you distribute to can activate their virtualenv and do:
$ pip install -r requirements.txt
Make sure your package will work from a fresh virtualenv by installing with that method. This is also good to check before installing via a setup.py. You want to make sure your requirements will get anyone working who just does a fresh sudo python setup.py install, or python setup.py install in a virtualenv context.
You can exit a virtualenv context with:
$ deactivate

Your best bet may be to include a copy of the protobuf runtime library with your package, maybe under a different package name. Then you can make sure that it matches the version of your generated code.
Another option is to invoke protoc as part of the installation process, so you get whatever version is available on the host.
I don't think packaging multiple versions of your generated code sounds like a good idea -- you'll just have problems again when the next protobuf release comes out.

How do you install pycairo on Mac OS X? [duplicate]

I am trying to install the pycairo (Python bindings for the cairo graphics library) under OSX.
I started with
easy_install pycairo
and got:
Requested 'cairo >= 1.8.8' but version of cairo is 1.0.4
error: Setup script exited with Error: cairo >= 1.8.8 not found
So I went to cairo's site and downloaded the latest package (1.8.8) of cairo, and also the latest package of something called pixman (both source packages -- couldn't find osx binaries)
unzipped both, each in own directory.
for pixman, the regular ./configure ; make ; sudo make install worked just find
for cairo, ./configure seemed to work, but make failed with:
In file included from cairo-analysis-surface.c:37:
cairoint.h:71:20: error: pixman.h: No such file or directory
What am I doing wrong?
And why do I have to struggle so much to get a software library to work on an os that "just works"? Why isn't darwin more like linux?

If you already have homebrew, these two commands should be helpful:
$ brew install cairo --use-clang
$ brew install py2cairo
For a non-Homebrew installed Python, set the PYTHONPATH to find pycairo. You can set your PYTHONPATH in your .bashrc/.profile/.whatever to the following:
PYTHONPATH=/usr/local/lib/python2.7/site-packages:$PYTHONPATH.
I personally didn't need to use this last part but it might help you.

It appears you are mixing various install options here. The MacPorts package system port install command should automatically pull in all the dependencies needed for a particular package so the trick is to start with the right top-level project. For python packages, MacPorts has a general convention currently: packages that start with py- are for python 2.4, those with py25- are for 2.5, and py26- for 2.6. There are currently py-cairo, py25-cairo, and py26-cairo packages available in MacPorts.
By choosing py-cairo you picked the python2.4 version and you'll probably find that MacPorts built and installed a python2.4 for you (linked at /opt/local/bin/python2.4) and, if you launch it, you'll probably find that you can import cairo there. Now that may be OK for your needs but Python 2.4 is quite old and no longer supported so, if you're just starting, it might be better to start with Python 2.6, one of the two current versions of Python. To do so, all you should need to do is:
sudo port install py26-cairo
That should bring in any missing dependencies, mainly the MacPorts python2.6, which you can run from /opt/local/bin/python2.6. You may want to change your $PATH in your shell startup script, probably .bash_profile, to put /opt/local/bin early on the search path.
Because installing Cairo and its python bindings seems to be fairly complex, it should be easier and better to stick to using a complete MacPorts solution for this. That does mean you've needlessly (and harmlessly) installed a couple of Python instances that you won't need. But if you do want to clean things up a bit, you can easily remove the MacPorts python24 with:
sudo port uninstall py-cairo python24
Completely removing the python.org installed python is more complicated. I've explained the process here. But there's no pressing need to remove either as long as you keep your paths straight.

Ok. I solved it. Putting solution here for future reference, it might help someone.
Basically, the whole ports/fink system is a bit messed up, and osx doesn't really play nice with the linux-y world.
So, the steps I needed to install pycairo on OSX were:
download the latest source versions of pixman, cairo, pycairo
extract everything. Then:
cd PIXMAN_DIR ; ./configure ; make ; sudo make install
cd CAIRO_DIR ; cp PIXMAN_DIR/pixman/*.h . ; ./configure ; make ; sudo make install
cd PYCAYRO_DIR; locate cairo.pc
hopefully, several locations are returned. choose the most likely one (one with newest cairo). For me it was "/opt/local/lib/pkgconfig/cairo.pc" and do:
export PKG_CONFIG_PATH=/opt/local/lib/pkgconfig/
after this, still in PYCAIRO_DIR, do:
python setup.py install
This should do it...

The port command installs the library for the darwinports python installation, which is different to the framework build (so steps 2 and 3 shouldn't work). Try sudo easy_install pycairo instead (although your step 4 should be equivalent to this).
Look at which python too, to check that you are in fact running the python you think you are.

On Mac OS you can have multiple Python versions installed. You can have even more if you decide to install Python via Fink or MacPorts. When you compile libraries from the source, you should make sure they point to the correct installation.
I currently have Python 2.5.1 and Python 2.6.4 installed on my machine, which I can call via python2.5 and python respectively. They live in two different folders:
/System/Library/Frameworks/Python.framework/Versions/2.5and /Library/Frameworks/Python.framework/Versions/2.6
I was running into a similar problem when compiling pycairo 1.8.8 from the tarball. The INSTALL file in this case is your friend, as it contains the correct instructions to avoid potential version conflicts. You basically need to specify the correct prefix so that the package will be installed in the correct folder.
$ python -c "import sys; print sys.prefix"
# make a note of the python prefix
$ ./configure --prefix=[python_prefix]
$ make
$ make install # may require superuser access
Running these instructions with python2.5 and python you will be able to correctly install pycairo for both versions (or for any version installed via MacPorts / Fink).

Step 1: Run this from terminal ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)" < /dev/null 2> /dev/null
Step 2: brew install cairo
Step 3: pip install pycairo

What is the most compatible way to install python modules on a Mac?

I'm starting to learn python and loving it. I work on a Mac mainly as well as Linux. I'm finding that on Linux (Ubuntu 9.04 mostly) when I install a python module using apt-get it works fine. I can import it with no trouble.
On the Mac, I'm used to using Macports to install all the Unixy stuff. However, I'm finding that most of the python modules I install with it are not being seen by python. I've spent some time playing around with PATH settings and using python_select . Nothing has really worked and at this point I'm not really understanding, instead I'm just poking around.
I get the impression that Macports isn't universally loved for managing python modules. I'd like to start fresh using a more "accepted" (if that's the right word) approach.
So, I was wondering, what is the method that Mac python developers use to manage their modules?
Bonus questions:
Do you use Apple's python, or some other version?
Do you compile everything from source or is there a package manger that works well (Fink?).

The most popular way to manage python packages (if you're not using your system package manager) is to use setuptools and easy_install. It is probably already installed on your system. Use it like this:
easy_install django
easy_install uses the Python Package Index which is an amazing resource for python developers. Have a look around to see what packages are available.
A better option is pip, which is gaining traction, as it attempts to fix a lot of the problems associated with easy_install. Pip uses the same package repository as easy_install, it just works better. Really the only time use need to use easy_install is for this command:
easy_install pip
After that, use:
pip install django
At some point you will probably want to learn a bit about virtualenv. If you do a lot of python development on projects with conflicting package requirements, virtualenv is a godsend. It will allow you to have completely different versions of various packages, and switch between them easily depending your needs.
Regarding which python to use, sticking with Apple's python will give you the least headaches, but If you need a newer version (Leopard is 2.5.1 I believe), I would go with the macports python 2.6.

Your question is already three years old and there are some details not covered in other answers:
Most people I know use HomeBrew or MacPorts, I prefer MacPorts because of its clean cut of what is a default Mac OS X environment and my development setup. Just move out your /opt folder and test your packages with a normal user Python environment
MacPorts is only portable within Mac, but with easy_install or pip you will learn how to setup your environment in any platform (Win/Mac/Linux/Bsd...). Furthermore it will always be more up to date and with more packages
I personally let MacPorts handle my Python modules to keep everything updated. Like any other high level package manager (ie: apt-get) it is much better for the heavy lifting of modules with lots of binary dependencies. There is no way I would build my Qt bindings (PySide) with easy_install or pip. Qt is huge and takes a lot to compile. As soon as you want a Python package that needs a library used by non Python programs, try to avoid easy_install or pip
At some point you will find that there are some packages missing within MacPorts. I do not believe that MacPorts will ever give you the whole CheeseShop. For example, recently I needed the Elixir module, but MacPorts only offers py25-elixir and py26-elixir, no py27 version. In cases like these you have:
pip-2.7 install --user elixir
( make sure you always type pip-(version) )
That will build an extra Python library in your home dir. Yes, Python will work with more than one library location: one controlled by MacPorts and a user local one for everything missing within MacPorts.
Now notice that I favor pip over easy_install. There is a good reason you should avoid setuptools and easy_install. Here is a good explanation and I try to keep away from them. One very useful feature of pip is giving you a list of all the modules (along their versions) that you installed with MacPorts, easy_install and pip itself:
pip-2.7 freeze
If you already started using easy_install, don't worry, pip can recognize everything done already by easy_install and even upgrade the packages installed with it.
If you are a developer keep an eye on virtualenv for controlling different setups and combinations of module versions. Other answers mention it already, what is not mentioned so far is the Tox module, a tool for testing that your package installs correctly with different Python versions.
Although I usually do not have version conflicts, I like to have virtualenv to set up a clean environment and get a clear view of my packages dependencies. That way I never forget any dependencies in my setup.py
If you go for MacPorts be aware that multiple versions of the same package are not selected anymore like the old Debian style with an extra python_select package (it is still there for compatibility). Now you have the select command to choose which Python version will be used (you can even select the Apple installed ones):
$ port select python
Available versions for python:
none
python25-apple
python26-apple
python27 (active)
python27-apple
python32
$ port select python python32
Add tox on top of it and your programs should be really portable

Please see Python OS X development environment. The best way is to use MacPorts. Download and install MacPorts, then install Python via MacPorts by typing the following commands in the Terminal:
sudo port install python26 python_select
sudo port select --set python python26
OR
sudo port install python30 python_select
sudo port select --set python python30
Use the first set of commands to install Python 2.6 and the second set to install Python 3.0. Then use:
sudo port install py26-packagename
OR
sudo port install py30-packagename
In the above commands, replace packagename with the name of the package, for example:
sudo port install py26-setuptools
These commands will automatically install the package (and its dependencies) for the given Python version.
For a full list of available packages for Python, type:
port list | grep py26-
OR
port list | grep py30-
Which command you use depends on which version of Python you chose to install.

I use MacPorts to install Python and any third-party modules tracked by MacPorts into /opt/local, and I install any manually installed modules (those not in the MacPorts repository) into /usr/local, and this has never caused any problems. I think you may be confused as to the use of certain MacPorts scripts and environment variables.
MacPorts python_select is used to select the "current" version of Python, but it has nothing to do with modules. This allows you to, e.g., install both Python 2.5 and Python 2.6 using MacPorts, and switch between installs.
The $PATH environment variables does not affect what Python modules are loaded. $PYTHONPATH is what you are looking for. $PYTHONPATH should point to directories containing Python modules you want to load. In my case, my $PYTHONPATH variable contains /usr/local/lib/python26/site-packages. If you use MacPorts' Python, it sets up the other proper directories for you, so you only need to add additional paths to $PYTHONPATH. But again, $PATH isn't used at all when Python searches for modules you have installed.
$PATH is used to find executables, so if you install MacPorts' Python, make sure /opt/local/bin is in your $PATH.

There's nothing wrong with using a MacPorts Python installation. If you are installing python modules from MacPorts but then not seeing them, that likely means you are not invoking the MacPorts python you installed to. In a terminal shell, you can use absolute paths to invoke the various Pythons that may be installed. For example:
$ /usr/bin/python2.5 # Apple-supplied 2.5 (Leopard)
$ /opt/local/bin/python2.5 # MacPorts 2.5
$ /opt/local/bin/python2.6 # MacPorts 2.6
$ /usr/local/bin/python2.6 # python.org (MacPython) 2.6
$ /usr/local/bin/python3.1 # python.org (MacPython) 3.1
To get the right python by default requires ensuring your shell $PATH is set properly to ensure that the right executable is found first. Another solution is to define shell aliases to the various pythons.
A python.org (MacPython) installation is fine, too, as others have suggested. easy_install can help but, again, because each Python instance may have its own easy_install command, make sure you are invoking the right easy_install.

If you use Python from MacPorts, it has it's own easy_install located at: /opt/local/bin/easy_install-2.6 (for py26, that is). It's not the same one as simply calling easy_install directly, even if you used python_select to change your default python command.

Have you looked into easy_install at all? It won't synchronize your macports or anything like that, but it will automatically download the latest package and all necessary dependencies, i.e.
easy_install nose
for the nose unit testing package, or
easy_install trac
for the trac bug tracker.
There's a bit more information on their EasyInstall page too.

For MacPython installations, I found an effective solution to fixing the problem with setuptools (easy_install) in this blog post:
http://droidism.com/getting-running-with-django-and-macpython-26-on-leopard
One handy tip includes finding out which version of python is active in the terminal:
which python

When you install modules with MacPorts, it does not go into Apple's version of Python. Instead those modules are installed onto the MacPorts version of Python selected.
You can change which version of Python is used by default using a mac port called python_select. instructions here.
Also, there's easy_install. Which will use python to install python modules.

You may already have pip3 pre-installed, so just try it!

Regarding which python version to use, Mac OS usually ships an old version of python. It's a good idea to upgrade to a newer version. You can download a .dmg from http://www.python.org/download/ . If you do that, remember to update the path. You can find the exact commands here http://farmdev.com/thoughts/66/python-3-0-on-mac-os-x-alongside-2-6-2-5-etc-/

I use easy_install with Apple's Python, and it works like a charm.

Directly install one of the fink packages (Django 1.6 as of 2013-Nov)
fink install django-py27
fink install django-py33
Or create yourself a virtualenv:
fink install virtualenv-py27
virtualenv django-env
source django-env/bin/activate
pip install django
deactivate # when you are done
Or use fink django plus any other pip installed packages in a virtualenv
fink install django-py27
fink install virtualenv-py27
virtualenv django-env --system-site-packages
source django-env/bin/activate
# django already installed
pip install django-analytical # or anything else you might want
deactivate # back to your normally scheduled programming