Develop Reference

Cannot install and upgrade python packages on Ubuntu 14.04

I am trying to install some python packages but I am getting this kind of error every time.
The directory '/home/username/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
The directory '/home/username/.cache/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Exception:
Traceback (most recent call last):
File "/usr/lib/python2.7/dist-packages/pip/basecommand.py", line 209, in main
status = self.run(options, args)
File "/usr/lib/python2.7/dist-packages/pip/commands/install.py", line 317, in run
requirement_set.prepare_files(finder)
File "/usr/lib/python2.7/dist-packages/pip/req/req_set.py", line 360, in prepare_files
ignore_dependencies=self.ignore_dependencies))
File "/usr/lib/python2.7/dist-packages/pip/req/req_set.py", line 512, in _prepare_file
finder, self.upgrade, require_hashes)
File "/usr/lib/python2.7/dist-packages/pip/req/req_install.py", line 273, in populate_link
self.link = finder.find_requirement(self, upgrade)
File "/usr/lib/python2.7/dist-packages/pip/index.py", line 442, in find_requirement
all_candidates = self.find_all_candidates(req.name)
File "/usr/lib/python2.7/dist-packages/pip/index.py", line 400, in find_all_candidates
for page in self._get_pages(url_locations, project_name):
File "/usr/lib/python2.7/dist-packages/pip/index.py", line 545, in _get_pages
page = self._get_page(location)
File "/usr/lib/python2.7/dist-packages/pip/index.py", line 648, in _get_page
return HTMLPage.get_page(link, session=self.session)
File "/usr/lib/python2.7/dist-packages/pip/index.py", line 757, in get_page
"Cache-Control": "max-age=600",
File "/usr/share/python-wheels/requests-2.9.1-py2.py3-none-any.whl/requests/sessions.py", line 480, in get
return self.request('GET', url, **kwargs)
File "/usr/lib/python2.7/dist-packages/pip/download.py", line 378, in request
return super(PipSession, self).request(method, url, *args, **kwargs)
File "/usr/share/python-wheels/requests-2.9.1-py2.py3-none-any.whl/requests/sessions.py", line 468, in request
resp = self.send(prep, **send_kwargs)
File "/usr/share/python-wheels/requests-2.9.1-py2.py3-none-any.whl/requests/sessions.py", line 576, in send
r = adapter.send(request, **kwargs)
File "/usr/share/python-wheels/CacheControl-0.11.5-py2.py3-none-any.whl/cachecontrol/adapter.py", line 46, in send
resp = super(CacheControlAdapter, self).send(request, **kw)
File "/usr/share/python-wheels/requests-2.9.1-py2.py3-none-any.whl/requests/adapters.py", line 376, in send
timeout=timeout
File "/usr/share/python-wheels/urllib3-1.13.1-py2.py3-none-any.whl/urllib3/connectionpool.py", line 610, in urlopen
_stacktrace=sys.exc_info()[2])
File "/usr/share/python-wheels/urllib3-1.13.1-py2.py3-none-any.whl/urllib3/util/retry.py", line 228, in increment
total -= 1
TypeError: unsupported operand type(s) for -=: 'Retry' and 'int'
I looked it up on this site and Stack Overflow and all the solutions suggest installing
wheel
or
setuptools
But I am getting
Requirement already satisfied
The same problem occurs for trying to upgrade python packages.
Some of the commands that give me this error are:
sudo pip install -U textblob
My pip version is pip 8.1.1
What should I do?

Try, as suggested by the error message, running:
sudo -H pip install -U textblob
It is also well worth making sure that your version of pip is up to date with:
sudo python -m pip install -U pip
or
sudo -H python -m pip install -U pip

This happens to me frequently. Try this:
pip install textblob --user
This will work fine with other packages as well.
Thanks !

'error: Error -5 while decompressing data: incomplete or truncated stream' when installing pip package

I'm having the following error when running pip install Pillow==2.9.0 in a virtualenv: error: Error -5 while decompressing data: incomplete or truncated stream
Other packages install/uninstall fine, it just seems to affects Pillow 2.9.0. It doesn't seem to matter what virtualenv I'm in (or not).
Downloading a source tarball and installing from that worked, but since this is on a build server that's not an ideal workaround as I want to rely on pip install -r requirements.txt
Versions:
pip --version: pip 7.1.0 from /usr/local/lib/python2.7/site-packages (python 2.7)
python --version: Python 2.7.10
The full traceback is:
Collecting Pillow==2.9.0
/mnt/jenkins/jobA/workspace/.pyenv/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
Exception:
Traceback (most recent call last):
File "/mnt/jenkins/jobA/workspace/.pyenv/local/lib/python2.7/site-packages/pip/basecommand.py", line 223, in main
status = self.run(options, args)
File "/mnt/jenkins/jobA/workspace/.pyenv/local/lib/python2.7/site-packages/pip/commands/install.py", line 282, in run
requirement_set.prepare_files(finder)
File "/mnt/jenkins/jobA/workspace/.pyenv/local/lib/python2.7/site-packages/pip/req/req_set.py", line 334, in prepare_files
functools.partial(self._prepare_file, finder))
File "/mnt/jenkins/jobA/workspace/.pyenv/local/lib/python2.7/site-packages/pip/req/req_set.py", line 321, in _walk_req_to_install
more_reqs = handler(req_to_install)
File "/mnt/jenkins/jobA/workspace/.pyenv/local/lib/python2.7/site-packages/pip/req/req_set.py", line 491, in _prepare_file
session=self.session)
File "/mnt/jenkins/jobA/workspace/.pyenv/local/lib/python2.7/site-packages/pip/download.py", line 825, in unpack_url
session,
File "/mnt/jenkins/jobA/workspace/.pyenv/local/lib/python2.7/site-packages/pip/download.py", line 673, in unpack_http_url
from_path, content_type = _download_http_url(link, session, temp_dir)
File "/mnt/jenkins/jobA/workspace/.pyenv/local/lib/python2.7/site-packages/pip/download.py", line 857, in _download_http_url
stream=True,
File "/mnt/jenkins/jobA/workspace/.pyenv/local/lib/python2.7/site-packages/pip/_vendor/requests/sessions.py", line 477, in get
return self.request('GET', url, **kwargs)
File "/mnt/jenkins/jobA/workspace/.pyenv/local/lib/python2.7/site-packages/pip/download.py", line 373, in request
return super(PipSession, self).request(method, url, *args, **kwargs)
File "/mnt/jenkins/jobA/workspace/.pyenv/local/lib/python2.7/site-packages/pip/_vendor/requests/sessions.py", line 465, in request
resp = self.send(prep, **send_kwargs)
File "/mnt/jenkins/jobA/workspace/.pyenv/local/lib/python2.7/site-packages/pip/_vendor/requests/sessions.py", line 573, in send
r = adapter.send(request, **kwargs)
File "/mnt/jenkins/jobA/workspace/.pyenv/local/lib/python2.7/site-packages/pip/_vendor/cachecontrol/adapter.py", line 36, in send
cached_response = self.controller.cached_request(request)
File "/mnt/jenkins/jobA/workspace/.pyenv/local/lib/python2.7/site-packages/pip/_vendor/cachecontrol/controller.py", line 102, in cached_request
resp = self.serializer.loads(request, self.cache.get(cache_url))
File "/mnt/jenkins/jobA/workspace/.pyenv/local/lib/python2.7/site-packages/pip/_vendor/cachecontrol/serialize.py", line 108, in loads
return getattr(self, "_loads_v{0}".format(ver))(request, data)
File "/mnt/jenkins/jobA/workspace/.pyenv/local/lib/python2.7/site-packages/pip/_vendor/cachecontrol/serialize.py", line 164, in _loads_v2
cached = json.loads(zlib.decompress(data).decode("utf8"))
error: Error -5 while decompressing data: incomplete or truncated stream

Turns out that there was a corrupt entry in pip's local cache (located in my case, and by default I believe, in ~/.cache/pip).
I tested that by trying pip install --no-cache-dir Pillow==2.9.0 and lo and behold, it worked.
To confirm it was the cache, I ran:
pip uninstall Pillow
rm -rf ~/.cache/pip/*
pip install Pillow==2.9.0
which succeeded where it had failed before.
I don't know how there came to be a problem with the cache, but my guess is that pip got interrupted mid-download causing the cached data for Pillow to be corrupted

I found my issue to be with memory, of the disk.
Running df showed I had used 92% memory. After deleting and cleaning the hard drive (using Disk Usage Analyzer) I was able to successfully decompress data

for anyone having similar situation as mine. It's run out of space during installation --> cache still there but corrupted.
so ?
remove this folder ~/.cache/pip and pip works again.

Why pip doesn't work in virtual enviroment?

I am trying to use pip from virtual enviroment on Linux Ubuntu 14.04 x64.
While using pip from shell as:
pip search bla
it works fine.
But using from virtualenv it shows error:
(proj_testing)uxu#box:~$ pip search bla
/home/uxu/.virtualenvs/proj_testing/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
/home/uxu/.virtualenvs/proj_testing/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
Exception:
Traceback (most recent call last):
File "/home/uxu/.virtualenvs/proj_testing/local/lib/python2.7/site-packages/pip/basecommand.py", line 223, in main
status = self.run(options, args)
File "/home/uxu/.virtualenvs/proj_testing/local/lib/python2.7/site-packages/pip/commands/search.py", line 43, in run
pypi_hits = self.search(query, options)
File "/home/uxu/.virtualenvs/proj_testing/local/lib/python2.7/site-packages/pip/commands/search.py", line 60, in search
hits = pypi.search({'name': query, 'summary': query}, 'or')
File "/usr/lib/python2.7/xmlrpclib.py", line 1233, in __call__
return self.__send(self.__name, args)
File "/usr/lib/python2.7/xmlrpclib.py", line 1587, in __request
verbose=self.__verbose
File "/home/uxu/.virtualenvs/proj_testing/local/lib/python2.7/site-packages/pip/download.py", line 785, in request
headers=headers, stream=True)
File "/home/uxu/.virtualenvs/proj_testing/local/lib/python2.7/site-packages/pip/_vendor/requests/sessions.py", line 508, in post
return self.request('POST', url, data=data, json=json, **kwargs)
File "/home/uxu/.virtualenvs/proj_testing/local/lib/python2.7/site-packages/pip/download.py", line 373, in request
return super(PipSession, self).request(method, url, *args, **kwargs)
File "/home/uxu/.virtualenvs/proj_testing/local/lib/python2.7/site-packages/pip/_vendor/requests/sessions.py", line 465, in request
resp = self.send(prep, **send_kwargs)
File "/home/uxu/.virtualenvs/proj_testing/local/lib/python2.7/site-packages/pip/_vendor/requests/sessions.py", line 573, in send
r = adapter.send(request, **kwargs)
File "/home/uxu/.virtualenvs/proj_testing/local/lib/python2.7/site-packages/pip/_vendor/cachecontrol/adapter.py", line 46, in send
resp = super(CacheControlAdapter, self).send(request, **kw)
File "/home/uxu/.virtualenvs/proj_testing/local/lib/python2.7/site-packages/pip/_vendor/requests/adapters.py", line 431, in send
raise SSLError(e, request=request)
SSLError: [Errno 1] _ssl.c:510: error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed
It doesn't make sense, anyone got idea?

Try execute pip search bla like superuser
For ubuntu the command is:
sudo pip search bla

From https://urllib3.readthedocs.org/en/latest/security.html:
Certain Python platforms (specifically, versions of Python earlier than 2.7.9) have restrictions in their ssl module that limit the configuration that urllib3 can apply. In particular, this can cause HTTPS requests that would succeed on more featureful platforms to fail, and can cause certain security features to be unavailable. If you encounter this warning, it is strongly recommended you upgrade to a newer Python version, or that you use pyOpenSSL as described in the OpenSSL / PyOpenSSL section.
You probably have a less recent version of python in your virtual environment. Perhaps this is why this error only occurs in the virtual environment.

Virtual env won't run - setuptools pip failed with error code 2

I thought maybe it's because there are spaces in the directory, specifically the folder "Program Files (x86)".
People have suggested this error is a known bug in the newer virtualenv versions but I don't know how to install older versions. Apparently version 1.10.1 works but I don't know how to install it..
I'm running windows 7,
pip 6.0.8,
Django 1.7.7,
virtualenv 12.0.7,
Python 3.4.3
When trying to run virtualenv env command this happens..
G:\Program Files (x86)\Python34>virtualenv env
Using base prefix 'G:\\Program Files (x86)\\Python34'
New python executable in env\Scripts\python.exe
Installing setuptools, pip...
Complete output from command "G:\Program Files (x8...v\Scripts\python.exe" -c "import sys, pip; sys...d\"] + sys.argv[1:]))" setuptools pip:
Ignoring indexes: https://pypi.python.org/simple
Collecting setuptools
This repository located at None is not a trusted host, if this repository is available via HTTPS it is recommend to use HTTPS instead, otherwise you may silence this warning with '--trusted-host None'.
DEPRECATION: Implicitly allowing locations which are not hosted at a secure origin is deprecated and will require the use of --trusted-host in the future.
This repository located at None is not a trusted host, if this repository is available via HTTPS it is recommend to use HTTPS instead, otherwise you may silence this warning with '--trusted-host None'.
This repository located at None is not a trusted host, if this repository is available via HTTPS it is recommend to use HTTPS instead, otherwise you may silence this warning with '--trusted-host None'.
This repository located at None is not a trusted host, if this repository is available via HTTPS it is recommend to use HTTPS instead, otherwise you may silence this warning with '--trusted-host None'.
This repository located at None is not a trusted host, if this repository is available via HTTPS it is recommend to use HTTPS instead, otherwise you may silence this warning with '--trusted-host None'.
This repository located at None is not a trusted host, if this repository is available via HTTPS it is recommend to use HTTPS instead, otherwise you may silence this warning with '--trusted-host None'.
This repository located at None is not a trusted host, if this repository is available via HTTPS it is recommend to use HTTPS instead, otherwise you may silence this warning with '--trusted-host None'.
This repository located at None is not a trusted host, if this repository is available via HTTPS it is recommend to use HTTPS instead, otherwise you may silence this warning with '--trusted-host None'.
This repository located at None is not a trusted host, if this repository is available via HTTPS it is recommend to use HTTPS instead, otherwise you may silence this warning with '--trusted-host None'.
Exception:
Traceback (most recent call last):
File "G:\Program Files (x86)\Python34\lib\site-packages\virtualenv_support\pip-6.0.8-py2.py3-none-any.whl\pip\basecommand.py", line 232, in main
status = self.run(options, args)
File "G:\Program Files (x86)\Python34\lib\site-packages\virtualenv_support\pip-6.0.8-py2.py3-none-any.whl\pip\commands\install.py", line 339, in run
requirement_set.prepare_files(finder)
File "G:\Program Files (x86)\Python34\lib\site-packages\virtualenv_support\pip-6.0.8-py2.py3-none-any.whl\pip\req\req_set.py", line 333, in prepare_files
upgrade=self.upgrade,
File "G:\Program Files (x86)\Python34\lib\site-packages\virtualenv_support\pip-6.0.8-py2.py3-none-any.whl\pip\index.py", line 351, in find_requirement
for page in self._get_pages(locations, req):
File "G:\Program Files (x86)\Python34\lib\site-packages\virtualenv_support\pip-6.0.8-py2.py3-none-any.whl\pip\index.py", line 566, in _get_pages
page = self._get_page(location, req)
File "G:\Program Files (x86)\Python34\lib\site-packages\virtualenv_support\pip-6.0.8-py2.py3-none-any.whl\pip\index.py", line 783, in _get_page
return HTMLPage.get_page(link, req, session=self.session)
File "G:\Program Files (x86)\Python34\lib\site-packages\virtualenv_support\pip-6.0.8-py2.py3-none-any.whl\pip\index.py", line 872, in get_page
"Cache-Control": "max-age=600",
File "G:\Program Files (x86)\Python34\lib\site-packages\virtualenv_support\pip-6.0.8-py2.py3-none-any.whl\pip\_vendor\requests\sessions.py", line 473, in get
return self.request('GET', url, **kwargs)
File "G:\Program Files (x86)\Python34\lib\site-packages\virtualenv_support\pip-6.0.8-py2.py3-none-any.whl\pip\download.py", line 365, in request
return super(PipSession, self).request(method, url, *args, **kwargs)
File "G:\Program Files (x86)\Python34\lib\site-packages\virtualenv_support\pip-6.0.8-py2.py3-none-any.whl\pip\_vendor\requests\sessions.py", line 461, in request
resp = self.send(prep, **send_kwargs)
File "G:\Program Files (x86)\Python34\lib\site-packages\virtualenv_support\pip-6.0.8-py2.py3-none-any.whl\pip\_vendor\requests\sessions.py", line 567, in send
adapter = self.get_adapter(url=request.url)
File "G:\Program Files (x86)\Python34\lib\site-packages\virtualenv_support\pip-6.0.8-py2.py3-none-any.whl\pip\_vendor\requests\sessions.py", line 646, in get_adapter
raise InvalidSchema("No connection adapters were found for '%s'" % url)
pip._vendor.requests.exceptions.InvalidSchema: No connection adapters were found for 'g:\Program'
----------------------------------------
...Installing setuptools, pip...done.
Traceback (most recent call last):
File "G:\Program Files (x86)\Python34\lib\runpy.py", line 170, in _run_module_as_main
"__main__", mod_spec)
File "G:\Program Files (x86)\Python34\lib\runpy.py", line 85, in _run_code
exec(code, run_globals)
File "G:\Program Files (x86)\Python34\Scripts\virtualenv.exe\__main__.py", line 9, in <module>
File "G:\Program Files (x86)\Python34\lib\site-packages\virtualenv.py", line 825, in main
symlink=options.symlink)
File "G:\Program Files (x86)\Python34\lib\site-packages\virtualenv.py", line 993, in create_environment
install_wheel(to_install, py_executable, search_dirs)
File "G:\Program Files (x86)\Python34\lib\site-packages\virtualenv.py", line 961, in install_wheel
'PIP_NO_INDEX': '1'
File "G:\Program Files (x86)\Python34\lib\site-packages\virtualenv.py", line 903, in call_subprocess
% (cmd_desc, proc.returncode))
OSError: Command "G:\Program Files (x8...v\Scripts\python.exe" -c "import sys, pip; sys...d\"] + sys.argv[1:]))" setuptools pip failed with error code 2

Uninstalled Python 3.4 and installed Python 2.7 to the default directory "C:/Python27". Reinstalled pip and virtualenv.
Now when creating a virtualenv everything works fine. I would guess it was because of the spaces in the directory!

pip install - Connection reset by peer

When I try to install lxml using pip I had the exception "Connection reset by peer":
Downloading/unpacking lxml
Downloading lxml-3.0.1.tar.gz (3.2Mb): 643Kb downloaded
Exception:
Traceback (most recent call last):
File "/home/dummyuser/work/virt-dev-env/local/lib/python2.7/site-packages/pip-1.1-py2.7.egg/pip/basecommand.py", line 104, in main
status = self.run(options, args)
File "/home/dummyuser/work/virt-dev-env/local/lib/python2.7/site-packages/pip-1.1-py2.7.egg/pip/commands/install.py", line 245, in run
requirement_set.prepare_files(finder, force_root_egg_info=self.bundle, bundle=self.bundle)
File "/home/dummyuser/work/virt-dev-env/local/lib/python2.7/site-packages/pip-1.1-py2.7.egg/pip/req.py", line 985, in prepare_files
self.unpack_url(url, location, self.is_download)
File "/home/dummyuser/work/virt-dev-env/local/lib/python2.7/site-packages/pip-1.1-py2.7.egg/pip/req.py", line 1109, in unpack_url
retval = unpack_http_url(link, location, self.download_cache, self.download_dir)
File "/home/dummyuser/work/virt-dev-env/local/lib/python2.7/site-packages/pip-1.1-py2.7.egg/pip/download.py", line 451, in unpack_http_url
download_hash = _download_url(resp, link, temp_location)
File "/home/dummyuser/work/virt-dev-env/local/lib/python2.7/site-packages/pip-1.1-py2.7.egg/pip/download.py", line 368, in _download_url
chunk = resp.read(4096)
File "/usr/lib/python2.7/socket.py", line 380, in read
data = self._sock.recv(left)
File "/usr/lib/python2.7/httplib.py", line 561, in read
s = self.fp.read(amt)
File "/usr/lib/python2.7/socket.py", line 380, in read
data = self._sock.recv(left)
error: [Errno 104] Connection reset by peer
This only happened when installing lxml, other modules got installed with pip no problems. Anybody had the same problem?

Try to choose another PyPI mirror, either directly:
pip install -i http://e.pypi.python.org/simple lxml
Or by letting pip find the best mirror:
pip install --use-mirrors lxml

It turns out the mirror I was using somehow is not accessible from the network. The way I got around with it is installing it via OS directly using:
$ apt-get install python-lxml
then copy it to my virtual env:
$ cp -r /usr/lib/python2.7/dist-packages/lxml* /home/dummyuser/work/virt-dev-env/local/lib/python2.7/site-packages/
I then have it in my virtual env:
$ pip freeze
........
lxml==2.3.2
........

In my case, was an ipv6 issue. Some mirrors still don't have full ipv6 access.
Disable ipv6 and try again.