Develop Reference

six cannot be imported/uninstalled/reinstalled/upgraded

I'm trying to run lldb on my MacBook terminal, but I get errors related to the six package:
File "/Applications/Xcode.app/Contents/SharedFrameworks/LLDB.framework/Resources/Python/lldb/__init__.py", line 98, in <module>
import six
ImportError: No module named six
Traceback (most recent call last):
File "<string>", line 1, in <module>
I tried to launch python from terminal and then execute import six, but still:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named six
Next step was to uninstall and reinstall six, but it seems like everything related to this package is corrupted for me:
$: pip uninstall six
Uninstalling six:
/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/six-1.4.1-py2.7.egg-info
Proceed (y/n)? y
Exception:
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pip/basecommand.py", line 122, in main
status = self.run(options, args)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pip/commands/uninstall.py", line 59, in run
requirement_set.uninstall(auto_confirm=options.yes)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pip/req.py", line 1035, in uninstall
req.uninstall(auto_confirm=auto_confirm)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pip/req.py", line 598, in uninstall
paths_to_remove.remove(auto_confirm)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pip/req.py", line 1836, in remove
renames(path, new_path)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pip/util.py", line 295, in renames
shutil.move(old, new)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 302, in move
copy2(src, real_dst)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 131, in copy2
copystat(src, dst)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 103, in copystat
os.chflags(dst, st.st_flags)
OSError: [Errno 1] Operation not permitted: '/var/folders/_c/lj7t1035563flfh7fy4m3gfh0000gn/T/pip-YQdNMp-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/six-1.4.1-py2.7.egg-info'
Storing debug log for failure in /Users/lab5/.pip/pip.log
$: pip install --ignore-installed six
Downloading/unpacking six
Cannot fetch index base URL https://pypi.python.org/simple/
Could not find any downloads that satisfy the requirement six
Cleaning up...
No distributions at all found for six
Storing debug log for failure in /Users/lab5/.pip/pip.log
I downloaded six and installed it from the tar.gz file, but it still seems like the old version is installed:
$: pip install --ignore-installed ~/Downloads/six-1.5.1.tar.gz
Unpacking ./Downloads/six-1.5.1.tar.gz
Running setup.py (path:/var/folders/_c/lj7t1035563flfh7fy4m3gfh0000gn/T/pip-zqLhAa-build/setup.py) egg_info for package from file:///Users/lab5/Downloads/six-1.5.1.tar.gz
no previously-included directories found matching 'documentation/_build'
Installing collected packages: six
Running setup.py install for six
no previously-included directories found matching 'documentation/_build'
Successfully installed six
Cleaning up...
$: pip show six
---
Name: six
Version: 1.4.1
Location: /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python
Requires:
Python version: Python 2.7.15
Pip version: pip 1.5.6 from /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages (python 2.7)
Do you have any recommendations for me how to get rid of those errors and run lldb properly? I took someone's suggestion to install python again (simply brew install python), but it was not helpful.

You are using python2, to use brew to install python2 you need to add #2
brew install python#2
I use the python2 installed with brew and was able to install six just fine. From your errors, it appears when you tried pip install six you got the OSError: [Errno 1] Operation not permitted:
Try running pip with sudo -H
sudo -H pip install six --upgrade

I'm still not sure what the reasons for the errors were, but I could get rid of them after upgrading my python version and I'm just fine with that.
To install a new python version:
brew install python
Then, I removed the link at /usr/local/bin/python and set it again to the new version:
rm /usr/local/bin/python
ln -s /usr/local/Cellar/python/3.7.0/bin/python3.7 /usr/local/bin/python
Then, installed six with the new version's pip:
python -m pip install six
(executing python -m enforces installing with the new pip, for the new python)
I'm not 100% cool with that solution, I still have to remove old versions to have a clean python installation, but at least I can work with lldb now, which was my main goal

Pointing to the correct Xcode installation using xcode-select -switch solved it for me.

pip upgrade uninstalled itself

I used pip today for the first time in a while and I got the helpful message
You are using pip version 8.1.1, however version 9.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
So, I went ahead and
pip install --upgrade pip
but things did not go according to plan...
Collecting pip
Downloading pip-9.0.1-py2.py3-none-any.whl (1.3MB)
100% |████████████████████████████████| 1.3MB 510kB/s
Installing collected packages: pip
Found existing installation: pip 8.1.1
Uninstalling pip-8.1.1:
Exception:
Traceback (most recent call last):
File "//anaconda/lib/python2.7/site-packages/pip/basecommand.py", line 209, in main
status = self.run(options, args)
File "//anaconda/lib/python2.7/site-packages/pip/commands/install.py", line 317, in run
prefix=options.prefix_path,
File "//anaconda/lib/python2.7/site-packages/pip/req/req_set.py", line 726, in install
requirement.uninstall(auto_confirm=True)
File "//anaconda/lib/python2.7/site-packages/pip/req/req_install.py", line 746, in uninstall
paths_to_remove.remove(auto_confirm)
File "//anaconda/lib/python2.7/site-packages/pip/req/req_uninstall.py", line 115, in remove
renames(path, new_path)
File "//anaconda/lib/python2.7/site-packages/pip/utils/__init__.py", line 267, in renames
shutil.move(old, new)
File "//anaconda/lib/python2.7/shutil.py", line 303, in move
os.unlink(src)
OSError: [Errno 13] Permission denied: '/anaconda/lib/python2.7/site-packages/pip-8.1.1.dist-info/DESCRIPTION.rst'
You are using pip version 8.1.1, however version 9.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
And now it seems that pip is completely gone from my computer:
$ pip
-bash: //anaconda/bin/pip: No such file or directory
Is pip really gone, that is, did it really uninstall and then fail to reinstall, or did something just get unlinked? How can I avoid this issue in the future? Because I can imagine I will need to upgrade pip again at some point...

Python comes with a module for installing pip without needing to pull anything from the internet called ensurepip. It's pretty straightforward to use, just run the following in a terminal:
python -m ensurepip
From there you can upgrade pip to the latest the standard way. Additional documentation is available here https://docs.python.org/3/library/ensurepip.html.

You can reinstall pip with conda:
conda install pip
Looks like you need to have root rights:
sudo conda install pip

You can use curl to reinstall pip via the Python Packaging Authority website:
curl https://bootstrap.pypa.io/get-pip.py | python

Jupyter install fails on Mac

I'm trying to install Jupyter on my Mac (OS X El Capitan) and I'm getting an error in response to:
sudo pip install -U jupyter
At first the download/install starts fine, but then I run into this:
Installing collected packages: six, singledispatch, certifi, backports-abc, tornado, jupyter-core, pyzmq, jupyter-client, functools32, jsonschema, nbformat, pygments, mistune, MarkupSafe, jinja2, nbconvert, path.py, pickleshare, simplegeneric, setuptools, gnureadline, appnope, ptyprocess, pexpect, ipython, ipykernel, terminado, notebook, ipywidgets, jupyter-console, qtconsole, jupyter
Found existing installation: six 1.4.1
DEPRECATION: Uninstalling a distutils installed project (six) has been deprecated and will be removed in a future version. This is due to the fact that uninstalling a distutils project will only partially uninstall the project.
Uninstalling six-1.4.1:
Exception:
Traceback (most recent call last):
File "/Library/Python/2.7/site-packages/pip/basecommand.py", line 209, in main
status = self.run(options, args)
File "/Library/Python/2.7/site-packages/pip/commands/install.py", line 317, in run
prefix=options.prefix_path,
File "/Library/Python/2.7/site-packages/pip/req/req_set.py", line 726, in install
requirement.uninstall(auto_confirm=True)
File "/Library/Python/2.7/site-packages/pip/req/req_install.py", line 746, in uninstall
paths_to_remove.remove(auto_confirm)
File "/Library/Python/2.7/site-packages/pip/req/req_uninstall.py", line 115, in remove
renames(path, new_path)
File "/Library/Python/2.7/site-packages/pip/utils/__init__.py", line 267, in renames
shutil.move(old, new)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 302, in move
copy2(src, real_dst)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 131, in copy2
copystat(src, dst)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 103, in copystat
os.chflags(dst, st.st_flags)
OSError: [Errno 1] Operation not permitted: '/tmp/pip-ByX5xW-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/six-1.4.1-py2.7.egg-info'
What can I do to fix this?

The default OS X Python that El Capitan ships with is unfortunately rather badly mispackaged (grrr, Apple). Not only have they shipped some rather strange 3rd party packages already installed, but strange (old) and beta versions of those packages. Furthermore, they are heavily system protected.
This makes the default python rather unfriendly to do work with (as you have found out). In your specific case, juypter would like to install a recent version of the six library, but the system installed version is a strange old one and will not let pip update it (jupyter requires an updated version).
In general, to alleviate all future headaches, I recommend getting a different distribution of python, and putting that on your path so it is your new default. There are a couple of choices; what's important is only using one at a time (otherwise they're apt to confuse eachother, or confuse you).
Python.org - From the Python developers themselves
Homebrew - a unixy package manager for OS X, which has a normally-functioning python package
Anaconda Python - A scientific python distribution, with many 'harder-to-install' scientific packages already available and 'just working' (including jupyter).
I recommend going with Anaconda for now, if you don't know what to choose.

Or you could just try
sudo pip install -U jupyter --upgrade --ignore-installed six

This command will install jupyter for the currently logged in user, without ignoring anything:
sudo pip install --user jupyter

Ivo's answer is correct - the best solution is to repair your python installation. An example using Homebrew (which I highly recommend) is below:
Install Homebrew:
/usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
brew update
brew upgrade
Install Python:
brew install python
or Python 3:
brew install python3
Upgrade/install:
pip install --upgrade pip setuptools
Install jupyter:
pip install jupyter
Note: you might need to use sudo -H with pip install

upgrade pip without root access

I am trying to use pip 8 in a super computer where I don't have root access. After downloading pip from:
https://pip.pypa.io/en/stable/installing/
I do:
myname#edison04:~/software/pip> python get-pip.py --prefix=~/local_python_lib/
Collecting pip
Using cached pip-8.0.3-py2.py3-none-any.whl
Collecting wheel
Using cached wheel-0.29.0-py2.py3-none-any.whl
Installing collected packages: pip, wheel
Found existing installation: pip 1.5.4
Uninstalling pip-1.5.4:
Exception:
Traceback (most recent call last):
File "/tmp/tmplu5E1W/pip.zip/pip/basecommand.py", line 209, in main
status = self.run(options, args)
File "/tmp/tmplu5E1W/pip.zip/pip/commands/install.py", line 317, in run
prefix=options.prefix_path,
File "/tmp/tmplu5E1W/pip.zip/pip/req/req_set.py", line 725, in install
requirement.uninstall(auto_confirm=True)
File "/tmp/tmplu5E1W/pip.zip/pip/req/req_install.py", line 752, in uninstall
paths_to_remove.remove(auto_confirm)
File "/tmp/tmplu5E1W/pip.zip/pip/req/req_uninstall.py", line 115, in remove
renames(path, new_path)
File "/tmp/tmplu5E1W/pip.zip/pip/utils/__init__.py", line 266, in renames
shutil.move(old, new)
File "/usr/common/usg/python/2.7.9/lib/python2.7/shutil.py", line 300, in move
rmtree(src)
File "/usr/common/usg/python/2.7.9/lib/python2.7/shutil.py", line 247, in rmtree
rmtree(fullname, ignore_errors, onerror)
File "/usr/common/usg/python/2.7.9/lib/python2.7/shutil.py", line 252, in rmtree
onerror(os.remove, fullname, sys.exc_info())
File "/usr/common/usg/python/2.7.9/lib/python2.7/shutil.py", line 250, in rmtree
os.remove(fullname)
OSError: [Errno 13] Permission denied: '/global/common/edison/usg/python/2.7.5/lib/python2.7/site-packages/pip-1.5.4-py2.7.egg/pip/__init__.py'
You are using pip version 1.5.4, however version 8.0.3 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
It tries to remove the already existing pip which I don't want to (and I don't have permissions to).... May I ask what can i do? Thank you.
A few more updates: adding --user does not help:
myname#edison04:~/software/pip> python get-pip.py --prefix=~/local_python_lib/ --user
ERROR: Can not combine '--user' and '--prefix' as they imply different installation locations
You are using pip version 1.5.4, however version 8.0.3 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
myname#edison04:~/software/pip> python get-pip.py --user
Collecting pip
Using cached pip-8.0.3-py2.py3-none-any.whl
Installing collected packages: pip
Successfully installed pip-1.5.4
You are using pip version 1.5.4, however version 8.0.3 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
This installs pip-1.5.4 on some location, however I want to use pip 8.0.3 instead... Further more, when i locate the installed pip, and try to check their version... I think it just links my local bin to the pip 1.5.4 global bin
key01027#edison04:~> find . -name "pip"
./software/pip
./local_python_lib/pip
./.local/bin/pip
./.local/edison/2.7.9/lib/python2.7/site-packages/pip
./.local/edison/2.7.9/bin/pip
./.cache/pip
key01027#edison04:~> cd .local/
key01027#edison04:~/.local> cd bin/
key01027#edison04:~/.local/bin> ls
pip pip2 pip2.7
key01027#edison04:~/.local/bin> ./pip --version
pip 1.5.4 from /global/common/edison/usg/python/2.7.5/lib/python2.7/site-packages/pip-1.5.4-py2.7.egg (python 2.7)
key01027#edison04:~/.local/bin> ./pip2 --version
pip 1.5.4 from /global/common/edison/usg/python/2.7.5/lib/python2.7/site-packages/pip-1.5.4-py2.7.egg (python 2.7)
key01027#edison04:~/.local/bin> ./pip2.7 --version
pip 1.5.4 from /global/common/edison/usg/python/2.7.5/lib/python2.7/site-packages/pip-1.5.4-py2.7.egg (python 2.7)
What could I do? Thank you
further, using virtual env also does not solve this problem:
After set up the virtualenv, it also does not help...
(dev)key01027#edison08:~/software> which pip
/global/homes/k/key01027/dev/bin/pip
(dev)key01027#edison08:~/software> pip install --upgrade pip
Cannot fetch index base URL https://pypi.python.org/simple/
Could not find any downloads that satisfy the requirement pip in /global/common/edison/usg/python/2.7.5/lib/python2.7/site-packages/pip-1.5.4-py2.7.egg
Downloading/unpacking pip
Cleaning up...
No distributions at all found for pip in /global/common/edison/usg/python/2.7.5/lib/python2.7/site-packages/pip-1.5.4-py2.7.egg
Storing debug log for failure in /global/homes/k/key01027/.pip/pip.log

Python Virtual Environments solve this problem by allowing you to segment off your project's dependencies from others on the same system.
Most locked down systems still provide virtualenv, if not - contact your system administrator.
http://docs.python-guide.org/en/latest/dev/virtualenvs/
https://pypi.python.org/pypi/virtualenv

Cannot upgrade Pip. Permission denied. Why?

I'm trying to upgrade pip on my iMac, but I keep getting a permission denied error. I'm the admin, so I don't know what I've done wrong.
iMac:~ me$ pip install --upgrade pip
You are using pip version 6.0.8, however version 7.1.2 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
Collecting pip from https://pypi.python.org/packages/py2.py3/p/pip/pip-7.1.2-py2.py3-none-any.whl#md5=5ff9fec0be479e4e36df467556deed4d
Downloading pip-7.1.2-py2.py3-none-any.whl (1.1MB)
100% |################################| 1.1MB 181kB/s
Installing collected packages: pip
Found existing installation: pip 6.0.8
Uninstalling pip-6.0.8:
Exception:
Traceback (most recent call last):
File "/Library/Python/2.7/site-packages/pip-6.0.8-py2.7.egg/pip/basecommand.py", line 232, in main
status = self.run(options, args)
File "/Library/Python/2.7/site-packages/pip-6.0.8-py2.7.egg/pip/commands/install.py", line 347, in run
root=options.root_path,
File "/Library/Python/2.7/site-packages/pip-6.0.8-py2.7.egg/pip/req/req_set.py", line 543, in install
requirement.uninstall(auto_confirm=True)
File "/Library/Python/2.7/site-packages/pip-6.0.8-py2.7.egg/pip/req/req_install.py", line 667, in uninstall
paths_to_remove.remove(auto_confirm)
File "/Library/Python/2.7/site-packages/pip-6.0.8-py2.7.egg/pip/req/req_uninstall.py", line 126, in remove
renames(path, new_path)
File "/Library/Python/2.7/site-packages/pip-6.0.8-py2.7.egg/pip/utils/__init__.py", line 316, in renames
shutil.move(old, new)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 300, in move
rmtree(src)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 247, in rmtree
rmtree(fullname, ignore_errors, onerror)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 252, in rmtree
onerror(os.remove, fullname, sys.exc_info())
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 250, in rmtree
os.remove(fullname)
OSError: [Errno 13] Permission denied: '/Library/Python/2.7/site-packages/pip-6.0.8-py2.7.egg/EGG-INFO/dependency_links.txt'
I don't know what I've done wrong here as an upgrade seem fairly straightforward.
Thanks,

pip installations require elevation
sudo pip install --upgrade pip
Just because you are currently using an admin account doesn't mean the command will run with elevation, unless you specify sudo

You have to use sudo since you are upgrading pip "globally". I prefer to install Homebrew and let Homebrew manage everything.
Install Xcode from the App Store
Install Homebrew - ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
Install Python - brew install python, which also installs pip.
With this setup, everything is installed locally, which doesn't require you to run sudo.
I hope this helps.

You can do one of two ways:
chown -R user /path/to/file a more detailed information can be found in this SO post, while it doesn't relate to the exact problem it deals with the same issue
Quick and dirty way just do pip install --upgrade pip as already mentioned. However, many users will suggest against that.