upgrade pip without root access - python

I am trying to use pip 8 in a super computer where I don't have root access. After downloading pip from:
https://pip.pypa.io/en/stable/installing/
I do:
myname#edison04:~/software/pip> python get-pip.py --prefix=~/local_python_lib/
Collecting pip
Using cached pip-8.0.3-py2.py3-none-any.whl
Collecting wheel
Using cached wheel-0.29.0-py2.py3-none-any.whl
Installing collected packages: pip, wheel
Found existing installation: pip 1.5.4
Uninstalling pip-1.5.4:
Exception:
Traceback (most recent call last):
File "/tmp/tmplu5E1W/pip.zip/pip/basecommand.py", line 209, in main
status = self.run(options, args)
File "/tmp/tmplu5E1W/pip.zip/pip/commands/install.py", line 317, in run
prefix=options.prefix_path,
File "/tmp/tmplu5E1W/pip.zip/pip/req/req_set.py", line 725, in install
requirement.uninstall(auto_confirm=True)
File "/tmp/tmplu5E1W/pip.zip/pip/req/req_install.py", line 752, in uninstall
paths_to_remove.remove(auto_confirm)
File "/tmp/tmplu5E1W/pip.zip/pip/req/req_uninstall.py", line 115, in remove
renames(path, new_path)
File "/tmp/tmplu5E1W/pip.zip/pip/utils/__init__.py", line 266, in renames
shutil.move(old, new)
File "/usr/common/usg/python/2.7.9/lib/python2.7/shutil.py", line 300, in move
rmtree(src)
File "/usr/common/usg/python/2.7.9/lib/python2.7/shutil.py", line 247, in rmtree
rmtree(fullname, ignore_errors, onerror)
File "/usr/common/usg/python/2.7.9/lib/python2.7/shutil.py", line 252, in rmtree
onerror(os.remove, fullname, sys.exc_info())
File "/usr/common/usg/python/2.7.9/lib/python2.7/shutil.py", line 250, in rmtree
os.remove(fullname)
OSError: [Errno 13] Permission denied: '/global/common/edison/usg/python/2.7.5/lib/python2.7/site-packages/pip-1.5.4-py2.7.egg/pip/__init__.py'
You are using pip version 1.5.4, however version 8.0.3 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
It tries to remove the already existing pip which I don't want to (and I don't have permissions to).... May I ask what can i do? Thank you.
A few more updates: adding --user does not help:
myname#edison04:~/software/pip> python get-pip.py --prefix=~/local_python_lib/ --user
ERROR: Can not combine '--user' and '--prefix' as they imply different installation locations
You are using pip version 1.5.4, however version 8.0.3 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
myname#edison04:~/software/pip> python get-pip.py --user
Collecting pip
Using cached pip-8.0.3-py2.py3-none-any.whl
Installing collected packages: pip
Successfully installed pip-1.5.4
You are using pip version 1.5.4, however version 8.0.3 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
This installs pip-1.5.4 on some location, however I want to use pip 8.0.3 instead... Further more, when i locate the installed pip, and try to check their version... I think it just links my local bin to the pip 1.5.4 global bin
key01027#edison04:~> find . -name "pip"
./software/pip
./local_python_lib/pip
./.local/bin/pip
./.local/edison/2.7.9/lib/python2.7/site-packages/pip
./.local/edison/2.7.9/bin/pip
./.cache/pip
key01027#edison04:~> cd .local/
key01027#edison04:~/.local> cd bin/
key01027#edison04:~/.local/bin> ls
pip pip2 pip2.7
key01027#edison04:~/.local/bin> ./pip --version
pip 1.5.4 from /global/common/edison/usg/python/2.7.5/lib/python2.7/site-packages/pip-1.5.4-py2.7.egg (python 2.7)
key01027#edison04:~/.local/bin> ./pip2 --version
pip 1.5.4 from /global/common/edison/usg/python/2.7.5/lib/python2.7/site-packages/pip-1.5.4-py2.7.egg (python 2.7)
key01027#edison04:~/.local/bin> ./pip2.7 --version
pip 1.5.4 from /global/common/edison/usg/python/2.7.5/lib/python2.7/site-packages/pip-1.5.4-py2.7.egg (python 2.7)
What could I do? Thank you
further, using virtual env also does not solve this problem:
After set up the virtualenv, it also does not help...
(dev)key01027#edison08:~/software> which pip
/global/homes/k/key01027/dev/bin/pip
(dev)key01027#edison08:~/software> pip install --upgrade pip
Cannot fetch index base URL https://pypi.python.org/simple/
Could not find any downloads that satisfy the requirement pip in /global/common/edison/usg/python/2.7.5/lib/python2.7/site-packages/pip-1.5.4-py2.7.egg
Downloading/unpacking pip
Cleaning up...
No distributions at all found for pip in /global/common/edison/usg/python/2.7.5/lib/python2.7/site-packages/pip-1.5.4-py2.7.egg
Storing debug log for failure in /global/homes/k/key01027/.pip/pip.log

Python Virtual Environments solve this problem by allowing you to segment off your project's dependencies from others on the same system.
Most locked down systems still provide virtualenv, if not - contact your system administrator.
http://docs.python-guide.org/en/latest/dev/virtualenvs/
https://pypi.python.org/pypi/virtualenv

Related

Pip3 on mac os x 12.5 does not work even with virtual environment activated

I currently cannot successfully use venv to install python packages on my mac via pip3. I am currently running 12.5 Monterey.
Without a venv activated, running which python3 yields
/Library/Frameworks/Python.framework/Versions/3.10/bin/python3
I installed Python3.10.6 from python.org, so this is not the system version of python that ships with osx 12.5. After activating a virtual environment:
$ source ~/py_envs/my_env/bin/activate
I can then run the following commands and get the following output:
$ which python3
/Users/admin/py_envs/my_env/bin/python3
$ which pip3
/Users/admin/py_envs/my_env/bin/pip3
So far, this all seems correct. However when I attempt to install a package via pip3 with the virtual environment activated, I get the following output:
$ pip3 install redis
Collecting redis
Using cached redis-4.3.4-py3-none-any.whl (246 kB)
Collecting deprecated>=1.2.3
Using cached Deprecated-1.2.13-py2.py3-none-any.whl (9.6 kB)
Collecting async-timeout>=4.0.2
Using cached async_timeout-4.0.2-py3-none-any.whl (5.8 kB)
Collecting packaging>=20.4
Using cached packaging-21.3-py3-none-any.whl (40 kB)
Collecting wrapt<2,>=1.10
Using cached wrapt-1.14.1-cp310-cp310-macosx_11_0_arm64.whl (35 kB)
Collecting pyparsing!=3.0.5,>=2.0.2
Using cached pyparsing-3.0.9-py3-none-any.whl (98 kB)
Installing collected packages: wrapt, pyparsing, async-timeout, packaging, deprecated, redis
Successfully installed async-timeout-4.0.2 deprecated-1.2.13 packaging-21.3 pyparsing-3.0.9 redis-4.3.4 wrapt-1.14.1
ERROR: Exception:
Traceback (most recent call last):
File "/Users/admin/py_envs/rec_env/lib/python3.10/site-packages/pip/_internal/cli/base_command.py", line 167, in exc_logging_wrapper
status = run_func(*args)
File "/Users/admin/py_envs/rec_env/lib/python3.10/site-packages/pip/_internal/cli/req_command.py", line 247, in wrapper
return func(self, options, args)
File "/Users/admin/py_envs/rec_env/lib/python3.10/site-packages/pip/_internal/commands/install.py", line 520, in run
self._handle_target_dir(
File "/Users/admin/py_envs/rec_env/lib/python3.10/site-packages/pip/_internal/commands/install.py", line 530, in _handle_target_dir
ensure_dir(target_dir)
File "/Users/admin/py_envs/rec_env/lib/python3.10/site-packages/pip/_internal/utils/misc.py", line 105, in ensure_dir
os.makedirs(path)
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/os.py", line 215, in makedirs
makedirs(head, exist_ok=exist_ok)
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/os.py", line 215, in makedirs
makedirs(head, exist_ok=exist_ok)
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/os.py", line 215, in makedirs
makedirs(head, exist_ok=exist_ok)
[Previous line repeated 2 more times]
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/os.py", line 225, in makedirs
mkdir(name, mode)
PermissionError: [Errno 1] Operation not permitted: '/System/Library/Frameworks/Python.framework'
[notice] A new release of pip available: 22.2.1 -> 22.2.2
[notice] To update, run: pip install --upgrade pip
So it appears the version of global version of python3 is being referenced, even if I've activated my virtual environment. I have tried running pip install --upgrade pip which yields similar results. I have also tried installing python3 via homebrew, but attempting to install pacakges with pip still results in the same error. So far, I have not been able to find a solution on SO or elsewhere. Does anyone have any insight on this issue? Thanks in advance.
try running type pip3 to see if it is aliased. if it is aliased unalias pip3. and try again.
if that fails I suggest using pyenv to manage your python versions. remove all versions of python installed via homebrew and use pyenv to install python3. hope this helps.
To upgrade the pre-installed Python3 on macOS to the latest or higher version(s), you can use pyenv, which can be installed using Homebrew:
brew install pyenv
pyenv install [Python version]
You can see the list of the Python versions by running
pyenv versions
and finally, to set a version as a default one, just run:
pyenv global [Python version]
To set up a path for pyenv in the terminal (in my case => zsh), I manually add the following lines to .zshrc
export PYENV_ROOT="$HOME/.pyenv"
export PATH="$PYENV_ROOT/bin:$PATH"
eval "$(pyenv init --path)"

problems installing pandas module with pip on mac

I have 2 python installations on my mac osx.
$ which python
/usr/bin/python
$ which python3
/Library/Frameworks/Python.framework/Versions/3.4/bin/python3
$ pip install --upgrade setuptools
Collecting setuptools
Downloading https://files.pythonhosted.org/packages/37/06/754589caf971b0d2d48f151c2586f62902d93dc908e2fd9b9b9f6aa3c9dd/setuptools-40.6.3-py2.py3-none-any.whl (573kB)
100% |████████████████████████████████| 655kB 1.5MB/s
Installing collected packages: setuptools
Found existing installation: setuptools 2.1
Uninstalling setuptools-2.1:
Successfully uninstalled setuptools-2.1
Successfully installed setuptools-40.6.3
$ which pip
/Library/Frameworks/Python.framework/Versions/3.4/bin/pip
$ pip -V
pip 18.1 from /Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/pip (python 3.4)
I can install modules with pip fine. for example
$ python3 -m pip install --user tensorflow
Successfully installed absl-py-0.6.1 astor-0.7.1 gast-0.2.0 grpcio-1.17.1 markdown-3.0.1 protobuf-3.6.1 tensorboard-1.12.1 tensorflow-1.12.0 termcolor-1.1.0 werkzeug-0.14.1
but when i try the same with pandas
$ python3 -m pip install --user pandas
it goes through till the end by collecting and downloading dependencies, but errors out with the following error.
Collecting pandas
Downloading https://files.pythonhosted.org/packages/08/01/803834bc8a4e708aedebb133095a88a4dad9f45bbaf5ad777d2bea543c7e/pandas-0.22.0.tar.gz (11.3MB)
100% |████████████████████████████████| 11.4MB 1.5MB/s
Installing build dependencies ... done
Complete output from command python setup.py egg_info:
Download error on https://pypi.org/simple/numpy/: [SSL: TLSV1_ALERT_PROTOCOL_VERSION] tlsv1 alert protocol version (_ssl.c:600) -- Some packages may not be found!
Couldn't find index page for 'numpy' (maybe misspelled?)
Download error on https://pypi.org/simple/: [SSL: TLSV1_ALERT_PROTOCOL_VERSION] tlsv1 alert protocol version (_ssl.c:600) -- Some packages may not be found!
No local packages or working download links found for numpy>=1.9.0
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/private/var/folders/d5/k1w6m6xj5g727x952sdkkc1r0000gn/T/pip-install-r4n38agj/pandas/setup.py", line 743, in <module>
**setuptools_kwargs)
File "/private/var/folders/d5/k1w6m6xj5g727x952sdkkc1r0000gn/T/pip-build-env-fqe1gkak/lib/python3.4/site-packages/setuptools/__init__.py", line 142, in setup
_install_setup_requires(attrs)
File "/private/var/folders/d5/k1w6m6xj5g727x952sdkkc1r0000gn/T/pip-build-env-fqe1gkak/lib/python3.4/site-packages/setuptools/__init__.py", line 137, in _install_setup_requires
dist.fetch_build_eggs(dist.setup_requires)
File "/private/var/folders/d5/k1w6m6xj5g727x952sdkkc1r0000gn/T/pip-build-env-fqe1gkak/lib/python3.4/site-packages/setuptools/dist.py", line 586, in fetch_build_eggs
replace_conflicting=True,
File "/private/var/folders/d5/k1w6m6xj5g727x952sdkkc1r0000gn/T/pip-build-env-fqe1gkak/lib/python3.4/site-packages/pkg_resources/__init__.py", line 780, in resolve
replace_conflicting=replace_conflicting
File "/private/var/folders/d5/k1w6m6xj5g727x952sdkkc1r0000gn/T/pip-build-env-fqe1gkak/lib/python3.4/site-packages/pkg_resources/__init__.py", line 1063, in best_match
return self.obtain(req, installer)
File "/private/var/folders/d5/k1w6m6xj5g727x952sdkkc1r0000gn/T/pip-build-env-fqe1gkak/lib/python3.4/site-packages/pkg_resources/__init__.py", line 1075, in obtain
return installer(requirement)
File "/private/var/folders/d5/k1w6m6xj5g727x952sdkkc1r0000gn/T/pip-build-env-fqe1gkak/lib/python3.4/site-packages/setuptools/dist.py", line 653, in fetch_build_egg
return cmd.easy_install(req)
File "/private/var/folders/d5/k1w6m6xj5g727x952sdkkc1r0000gn/T/pip-build-env-fqe1gkak/lib/python3.4/site-packages/setuptools/command/easy_install.py", line 673, in easy_install
raise DistutilsError(msg)
distutils.errors.DistutilsError: Could not find suitable distribution for Requirement.parse('numpy>=1.9.0')
----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /private/var/folders/d5/k1w6m6xj5g727x952sdkkc1r0000gn/T/pip-install-r4n38agj/pandas/
The issue seems to be that it is using Python where it should be using python3 in python setup.py egg_info
how do i make sure that pip uses python3 in the scripts or if there is any other issues ?
You can use any code given below
pip3 install pandas --no-build-isolation
OR
sudo apt-get install python3-pip
sudo -H pip3 install pandas
OR
sudo pip install pandas
OR
pip install --user pandas
and for mac terminal using mojave works like
pip3 install pandas
or
pip 3 install --upgrade pip
after
pip install pandas

pip upgrade uninstalled itself

I used pip today for the first time in a while and I got the helpful message
You are using pip version 8.1.1, however version 9.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
So, I went ahead and
pip install --upgrade pip
but things did not go according to plan...
Collecting pip
Downloading pip-9.0.1-py2.py3-none-any.whl (1.3MB)
100% |████████████████████████████████| 1.3MB 510kB/s
Installing collected packages: pip
Found existing installation: pip 8.1.1
Uninstalling pip-8.1.1:
Exception:
Traceback (most recent call last):
File "//anaconda/lib/python2.7/site-packages/pip/basecommand.py", line 209, in main
status = self.run(options, args)
File "//anaconda/lib/python2.7/site-packages/pip/commands/install.py", line 317, in run
prefix=options.prefix_path,
File "//anaconda/lib/python2.7/site-packages/pip/req/req_set.py", line 726, in install
requirement.uninstall(auto_confirm=True)
File "//anaconda/lib/python2.7/site-packages/pip/req/req_install.py", line 746, in uninstall
paths_to_remove.remove(auto_confirm)
File "//anaconda/lib/python2.7/site-packages/pip/req/req_uninstall.py", line 115, in remove
renames(path, new_path)
File "//anaconda/lib/python2.7/site-packages/pip/utils/__init__.py", line 267, in renames
shutil.move(old, new)
File "//anaconda/lib/python2.7/shutil.py", line 303, in move
os.unlink(src)
OSError: [Errno 13] Permission denied: '/anaconda/lib/python2.7/site-packages/pip-8.1.1.dist-info/DESCRIPTION.rst'
You are using pip version 8.1.1, however version 9.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
And now it seems that pip is completely gone from my computer:
$ pip
-bash: //anaconda/bin/pip: No such file or directory
Is pip really gone, that is, did it really uninstall and then fail to reinstall, or did something just get unlinked? How can I avoid this issue in the future? Because I can imagine I will need to upgrade pip again at some point...
Python comes with a module for installing pip without needing to pull anything from the internet called ensurepip. It's pretty straightforward to use, just run the following in a terminal:
python -m ensurepip
From there you can upgrade pip to the latest the standard way. Additional documentation is available here https://docs.python.org/3/library/ensurepip.html.
You can reinstall pip with conda:
conda install pip
Looks like you need to have root rights:
sudo conda install pip
You can use curl to reinstall pip via the Python Packaging Authority website:
curl https://bootstrap.pypa.io/get-pip.py | python

On OS X El Capitan I can not upgrade a python package dependent on the six compatibility utilities NOR can I remove six

I am trying to use scrape, but I have a problem.
from six.moves import xmlrpc_client as xmlrpclib
ImportError: cannot import name xmlrpc_client
Then, I tried pip install --upgrade six scrape, but:
Found existing installation: six 1.4.1
DEPRECATION: Uninstalling a distutils installed project (six) has been deprecated and will be removed in a future version. This is due to the fact that uninstalling a distutils project will only partially uninstall the project.
Uninstalling six-1.4.1:
Exception:
Traceback (most recent call last):
File "/Library/Python/2.7/site-packages/pip/basecommand.py", line 211, in main
status = self.run(options, args)
File "/Library/Python/2.7/site-packages/pip/commands/install.py", line 311, in run
root=options.root_path,
File "/Library/Python/2.7/site-packages/pip/req/req_set.py", line 640, in install
requirement.uninstall(auto_confirm=True)
File "/Library/Python/2.7/site-packages/pip/req/req_install.py", line 716, in uninstall
paths_to_remove.remove(auto_confirm)
File "/Library/Python/2.7/site-packages/pip/req/req_uninstall.py", line 125, in remove
renames(path, new_path)
File "/Library/Python/2.7/site-packages/pip/utils/__init__.py", line 315, in renames
shutil.move(old, new)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 302, in move
copy2(src, real_dst)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 131, in copy2
copystat(src, dst)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 103, in copystat
os.chflags(dst, st.st_flags)
OSError: [Errno 1] Operation not permitted: '/var/folders/3h/r_2cxlvd1sjgzfgs4xckc__c0000gn/T/pip-5h86J8-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/six-1.4.1-py2.7.egg-info'
Quick Fix:
I just got around what I think was the same problem. You might consider trying this (sudo, if necessary):
pip install scrape --upgrade --ignore-installed six
Github is ultimately where I got this answer (and there are a few more suggestions you may consider if this one doesn't solve your problem). It also seems as though this is an El Capitan problem.
Also, this technically might be a duplicate. But the answer the other post came up with was installing your own Python rather than relying on the default osx Python, which strikes me as more laborious.
Longer, Real Fix:
Eventually I ran into a situation where I actually needed to upgrade six in order to install some other libraries, at which point ignoring wasn't enough. The easiest way to upgrade six for me was to not use pip, but instead to manually download the .tar.gz from the six page, decompress, move to the newly decompressed six package directory, and run the installation manually (sudo, if necessary):
python setup.py install
Thanks to this answer for the guidance.
I don't think this is a duplicate, but actually this issue discussed here on the pip GitHub repository issues list.
NOTE: Depending on which package you're installing, you may need to modify the command referenced in the solution. In my case below, I was trying to setup virtualenv and virtualwrapper. If you're failing on installing aws-cli or any other package, just insert that into the script
The core of the problem is tied to Apple's new SIP that they shipped with El Capitan. More specifically,
OS X 10.11's python retains its own copy of six which is unremoveable, because of modifications Apple has done to their python distribution. 1.4.1 is not the latest, 1.10.0 is. It also comes early on their python's import path, so it will typically override later versions you install.
I would suggest using a different python for now. Python.org's, or installed via Homebrew, or Anaconda Python.
There is an incredibly detailed discussion on the Ask Different Stack Exchange that covers how the problems with SIP have been identified, addressed, and evolved since the original release of El Capitan. Although I found it fascinating, you'll spend less time following the instructions below than it would take you to read it, so I'd reccomend checking it out AFTER you finish the following...
I ran into the exact same error when attempting to upgrade VirtualEnv & VirtualEnvWrapper. There were several suggestions kicked around on that above thread, but in the end the most stable was to
Leverage the built-in support for the sudo OPTION to specify a HOME environment variable
$ man sudo
-H The -H (HOME) option option sets the HOME environment variable
to the home directory of the target user (root by default) as specified
HOME environment variable depends on sudoers(5) settings. By default,
sudo will set HOME if env_reset or always_set_home are set, or if
set_home is set and the -s option is specified on the command line.
Leverage pip's options to force an upgrade and ignore any pre-existing packages
$ pip install --help | grep upgrade
-U, --upgrade Upgrade all specified packages to the newest available
version. This process is recursive regardless of whether a dependency
is already satisfied.
beejhuff#ignatius:~/mac_setup$ pip install --help | grep ignore-installed
-I, --ignore-installed Ignore the installed packages (reinstalling instead).
First, my original attempt & error:
$ sudo pip install virtualenv virtualenvwrapper
The directory '/Users/beejhuff/Library/Caches/pip/http' or its parent directory
is not owned by the current user and the cache has been disabled.
Please check the permissions and owner of that directory. If executing
pip with sudo, you may want sudo's -H flag.
The directory '/Users/beejhuff/Library/Caches/pip' or its parent directory
is not owned by the current user and caching wheels has been disabled.
check the permissions and owner of that directory. If executing pip with
sudo, you may want sudo's -H flag.
Collecting virtualenv
Downloading virtualenv-15.0.0-py2.py3-none-any.whl (1.8MB)
100% |████████████████████████████████| 1.8MB 335kB/s
Collecting virtualenvwrapper
Downloading virtualenvwrapper-4.7.1-py2.py3-none-any.whl
Collecting virtualenv-clone (from virtualenvwrapper)
Downloading virtualenv-clone-0.2.6.tar.gz
Collecting stevedore (from virtualenvwrapper)
Downloading stevedore-1.12.0-py2.py3-none-any.whl
Collecting pbr>=1.6 (from stevedore->virtualenvwrapper)
Downloading pbr-1.8.1-py2.py3-none-any.whl (89kB)
100% |████████████████████████████████| 92kB 362kB/s
Collecting six>=1.9.0 (from stevedore->virtualenvwrapper)
Downloading six-1.10.0-py2.py3-none-any.whl
Installing collected packages: virtualenv, virtualenv-clone, pbr, six, stevedore, virtualenvwrapper
Running setup.py install for virtualenv-clone ... done
Found existing installation: six 1.4.1
DEPRECATION: Uninstalling a distutils installed project (six) has been deprecated and will be removed in a future version. This is due to the fact that uninstalling a distutils project will only partially uninstall the project.
Uninstalling six-1.4.1:
Exception:
Traceback (most recent call last):
File "/Library/Python/2.7/site-packages/pip-8.1.0-py2.7.egg/pip/basecommand.py", line 209, in main
status = self.run(options, args)
File "/Library/Python/2.7/site-packages/pip-8.1.0-py2.7.egg/pip/commands/install.py", line 317, in run
prefix=options.prefix_path,
File "/Library/Python/2.7/site-packages/pip-8.1.0-py2.7.egg/pip/req/req_set.py", line 726, in install
requirement.uninstall(auto_confirm=True)
File "/Library/Python/2.7/site-packages/pip-8.1.0-py2.7.egg/pip/req/req_install.py", line 746, in uninstall
paths_to_remove.remove(auto_confirm)
File "/Library/Python/2.7/site-packages/pip-8.1.0-py2.7.egg/pip/req/req_uninstall.py", line 115, in remove
renames(path, new_path)
File "/Library/Python/2.7/site-packages/pip-8.1.0-py2.7.egg/pip/utils/__init__.py", line 267, in renames
shutil.move(old, new)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 302, in move
copy2(src, real_dst)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 131, in copy2
copystat(src, dst)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 103, in copystat
os.chflags(dst, st.st_flags)
OSError: [Errno 1] Operation not permitted: '/tmp/pip-GQL8Gi-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/six-1.4.1-py2.7.egg-info'
The Solution
It required modifying my installation command in THREE specific ways:
I had to add the -H flag to sudo
I had to add the --upgrade option AFTER the name of the package I was upgrading (virtualenv)
I had to use the --ignore-installed flag and specify the six package was the one to be ignored.
*Note: general command is $ sudo -H pip install <packagename> --upgrade --ignore-installed six - replace<packagename> with the specific package you need to install`
Final Working Example
1st Upgrade virtualenv
$ sudo -H pip install virtualenv --upgrade --ignore-installed six
Password:
Collecting virtualenv
Using cached virtualenv-15.0.0-py2.py3-none-any.whl
Collecting six
Using cached six-1.10.0-py2.py3-none-any.whl
Installing collected packages: virtualenv, six
Successfully installed six-1.4.1 virtualenv-15.0.0
2nd Upgrade virtualenvwrapper
$ sudo -H pip install virtualenvwrapper --upgrade --ignore-installed six
Password:
Downloading virtualenvwrapper-4.7.1-py2.py3-none-any.whl
Collecting six
Downloading six-1.10.0-py2.py3-none-any.whl
Collecting virtualenv (from virtualenvwrapper)
Downloading virtualenv-15.0.0-py2.py3-none-any.whl (1.8MB)
100% |████████████████████████████████| 1.8MB 751kB/s
Collecting virtualenv-clone (from virtualenvwrapper)
Downloading virtualenv-clone-0.2.6.tar.gz
Collecting stevedore (from virtualenvwrapper)
Downloading stevedore-1.12.0-py2.py3-none-any.whl
Collecting pbr>=1.6 (from stevedore->virtualenvwrapper)
Downloading pbr-1.8.1-py2.py3-none-any.whl (89kB)
100% |████████████████████████████████| 92kB 417kB/s
Installing collected packages: virtualenv, virtualenv-clone, pbr, six, stevedore, virtualenvwrapper
Running setup.py install for virtualenv-clone ... done
Successfully installed pbr-1.8.1 six-1.4.1 stevedore-1.12.0 virtualenv-15.0.0 virtualenv-clone-0.2.6 virtualenvwrapper-4.7.1
Both of previous answers don't work for me. Finally, I got the solution from GitHub, aws/aws-cli, as excerpted below.
On OS X, if you see an error regarding the version of six that came
with distutils in El Capitan, use the --ignore-installed option:
$ sudo pip install awscli --ignore-installed six
This would work!!
sudo pip install awsebcli --upgrade --ignore-installed six

Using --prefix when upgrading python packages

The --prefix option to setup.py (and pip via --install-option) seems broken.
I need to maintain a shared repository of updated python packages on a machine where I do not have sudo access. Theoretically the --prefix option to setup.py (and pip the via --install-option) can do what I need.
However if there is already a version of the package I am installing in the system directories, I cannot install it on my private repository. Installing without --upgrade fails because a package already exists in the system, and installing with --upgrade fails because the old package version cannot be removed. This is shown in the error message below.
Is there any way to prevent install from trying to remove the old version of a package? OR, more likely, am I just not understanding how this is supposed to work?
$ pip --version
pip 1.5.6 from /python-local/lib/python2.7/site-packages (python 2.7)
$ pip install -U --install-option="--prefix=/python-local" virtualenv
Downloading/unpacking virtualenv from https://pypi.python.org/packages/py2.py3/v/virtualenv/virtualenv-1.11.6-py2.py3-none-any.whl#md5=c2aa2fc4986967815f8cb3a3418e6031
Downloading virtualenv-1.11.6-py2.py3-none-any.whl (1.6MB): 1.6MB downloaded
Installing collected packages: virtualenv
Found existing installation: virtualenv 1.7.1.2
Uninstalling virtualenv:
Cleaning up...
Exception:
Traceback (most recent call last):
File "/python-local/lib/python2.7/site-packages/pip/basecommand.py", line 122, in main
status = self.run(options, args)
File "/python-local/lib/python2.7/site-packages/pip/commands/install.py", line 283, in run
requirement_set.install(install_options, global_options, root=options.root_path)
File "/python-local/lib/python2.7/site-packages/pip/req.py", line 1431, in install
requirement.uninstall(auto_confirm=True)
File "/python-local/lib/python2.7/site-packages/pip/req.py", line 598, in uninstall
paths_to_remove.remove(auto_confirm)
File "/python-local/lib/python2.7/site-packages/pip/req.py", line 1836, in remove
renames(path, new_path)
File "/python-local/lib/python2.7/site-packages/pip/util.py", line 295, in renames
shutil.move(old, new)
File "/usr/lib/python2.7/shutil.py", line 300, in move
os.unlink(src)
OSError: [Errno 13] Permission denied: '/usr/bin/virtualenv'
Storing debug log for failure in /home/user/.pip/pip.log
The answer is to use pip's --ignore-installed option. You must use this option whenever a read-only version of the package you are installing is visible in the current environment. This includes when using --user, --local, or --prefix, and also if you are installing into a virtualenv that was created with --system-site-packages
You can find this option mentioned at the very end of this section http://pip.readthedocs.org/en/latest/user_guide.html#user-installs
or with pip install --help

Categories

Resources