Develop Reference

How do I install Apache Airflow on a Mac running Big Sur 11.0.1?

When I run pip3 install apache-airflow (I used brew install python to install the latest version of Python locally first), I get a number of errors while building wheels for psutil and setproctitle ending with the following:
ERROR: Could not build wheels for setproctitle which use PEP 517 and cannot be installed directly
Running the command with the --use-deprecated legacy-resolver indicated in the note on the start page has the same result.
What else am I missing that is preventing a successful install on Airflow on my machine?

Which python version do you have?
And what airflow version do you wish to install?
Since you said you've upgraded to the latest python version, I'm assuming you have 3.9.1. on your local machine.
Airflow is not supporting python 3.9 so you will need to use an older version of python.
I used python 3.7 to install airflow 1.10.10 (after which you can patch up to the latest version 2.0).
If using pip does not work (like what happened to me), try using pyenv to install python earlier versions. You might also (re)install zlib to get it to work brew reinstall zlib bzip2
Since I had no pyenv installed, I had to go through the pain of upgrading homebrew to be able to do this (see steps here in case you need it)

How to use Homebrew to upgrade to a specific Python version?

I've been using Homebrew to install various packages on my Mac. Currently I have Python 2.7.13 installed via Homebrew but I'd like to upgrade to Python 3.5.x, not 3.6 which is brew's current default. At first, I just tried upgrading to Python 3:
brew install python3
Brew said "python 2.7.13 is already installed. To upgrade to 3.6.5, run brew upgrade python" which isn't the version I want.
I then tried to search to see what versions of Python brew has available:
brew search python
Now Homebrew tells me, "If you meant "python" specifically: It was migrated from caskroom/cask to homebrew/core."
I then looked at homebrew-core on Github but it doesn't appear to provide any instructions on how to do what I want to do. Does anyone know how to now display a list of Python versions that one can install using Homebrew and what command to use to install a specific version?

There are several discussions of this, here and elsewhere. There is no direct way to do what you want. The recommended approach is to install pyenv via brew, and use pyenv to manage the different versions of Python on your system.
The github repo has a very detailed and clear guide on usage:
https://github.com/pyenv/pyenv

To see the version of python use python --version, and for upgrade use sudo apt-get upgrade python.
I used this, and it's worked

Which version of Pip to use with my Python installs?

Being new to Python, I'd love to clear up a few points that I couldn't get from reading various articles and tutorials.
After using Homebrew to install Python3, I noticed that it had installed both Python3 and Python3.4. I was also a little surprised that there are now three versions of pip on my machine too; pip, pip3 and pip3.4.
I created a new virtualenv and told it to use Python3, using the following command:
virtualenv -p /usr/local/bin/python3 mysite
I was also surprised that the version of Python that it installed in my VM was 3.4.
Is it safe to have these multiple version of Python and Pip hanging around on my machine?
Am I right to assume that I should take extra care to use the matching version of pip with Python, for example, pip3.4 with Python3.4?

Yes, it is safe. Python uses this naming like python3.4, python3.5 etc to differentiate between releases. python3 is a symbolic link to the current python3.x version. Pip follows the same convention.
If you're using python3.4 explicitly, you should be using pip3.4 specifically as well. Otherwise, just use python3 and pip3. For Python 2, you can simply use python (which, unless you installed the Homebrew version as well), will be the system Python), and ditto for pip. python2.7 and pip2.7 may also work.
In general, to find out which Python version goes with which pip you're using, try:
pip --version
and you'll see the Python included in the result.

No need to worried about if you have multiple version of Python and Pip installed. just check your version by writing in terminal :
$ brew info python
or to check the version of pip write in terminal :
$ brew info pip
and make sure you have updated your both pip and python version (write in terminal $ brew upgrade pip/python)
and other way to install python is go to https://www.python.org/downloads/ and choose as your requirement, there is two version available 2.7.9 & 3.4.3 ,
after installing python write in terminal $ python -V to check its version :) Hope it will help :)

Install Pillow on 3.3 when 2.7 is also installed(Linux)

I have both Python 2.7 and 3.3 installed on my system. I'm trying to hopefully write everything for 3.3. I've run into a snag. I need to install Pillow on 3.3 so I can use Images. How do I get it to install on 3.3. If I try
pip install Pillow
It comes up and says 'Requirement already satisfied(use --upgrade to upgrade): Pillow in /usr/lib/python2.7/dist-packages'
How do I get Pillow to install on 3.3, since that is where I need it?
I ran through the rest of the install(setuptools and prerequisites). Not sure where they installed. I did use the python3-setuptools for the initial install attempt. I don't know if that helped to control the flow of the prerequisites to 3.3 or not.
This is on a Linux system.

Your 2.7 and 3.3 have their own separate site-packages locations.
And, just as they have their own separate executables (usually python and python2.7 for 2.7, and python3 and python3.3 for 3.3), when you install pip for each one, they'll each have their own pip scripts (usually pip, pip2, and pip2.7 vs. pip3 and pip3.3).
So, just do this:
pip3.3 install Pillow
As a side note, if you use virtual environments, either via the third-party virtualenv package or the stdlib venv package (3.3+ only, and really not worth using until 3.4), this problem goes away: when you're inside a virtual environment, it's either a 2.7 environment or a 3.3 environment, and it's as if nothing else exists.
The basic design is explained in PEP 394. How pip fits into that design is explained in… as far as I know, docs that haven't been written yet, but will hopefully be part of Python 3.4.0 and/or pip 1.5.something.
Older versions of pip (I believe before 1.5.0) would use pip and pip-2.7 vs. pip and pip-3.3, which obviously leads to a bit of confusion.
And some distros have their own python-pip packages that do things differently. And then there's Arch, where python actually means 3.2. And so on.
But the basic idea is that when you have Python X.Y and V.W side by side, there will be some way to differentiate explicitly.

Python 2.6 in mac corrupted because of failled update

I'm a mac newbie and I tried to update my python version from 2.6 to 2.7. Unsuccessful, I changed my mind and uninstalled the python2.7 I had. I had a previous issue that if I typed something like:
python setup.py install
It would not install the package for python2.6, installing to the removed 2.7 version instead, to make it work I have to put
python2.6 setup.py install
And now when I try to install something with easy_install or pip (by the way, pip I have installed after 2.7 issue) I got the following huge message errors: here and here. I want to know how can I clean up my mess.

Since you were trying to install MySQLdb, how about you give ActivePython a try?
Install ActivePython 2.7 (it co-exists with Apple's System Python 2.6)
Open Terminal and type pypm install mysql-python (see PyPM Index) .. no compilation required
Make sure that /usr/local/bin/ is in front of your $PATH.
To uninstall ActivePython, you can do:
sudo /Library/Frameworks/Python.framework/Versions/2.7/Resources/Scripts/uninstall
Or, use sudo pythonselect 2.6 to switch the default Python in /usr/local/bin (if you have multiple versions of non-System Pythons installed)