Develop Reference

How to maintain same permissions on python dist-package after upgrade?

I am using a python package patroni version 1.0 on Ubuntu 14. On doing ls -lrt /usr/local/lib/python2.7/dist-packages/ I am seeing the permission like below
But once I upgrade the patroni via the command sudo pip install patroni --upgrade I am seeing the permission of the patroni changes after upgrade
The installation output is -
sudo pip install patroni --upgrade
The directory '/home/ubuntu/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
The directory '/home/ubuntu/.cache/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
/usr/local/lib/python2.7/dist-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
Collecting patroni
Requirement already up-to-date: requests in /usr/local/lib/python2.7/dist-packages (from patroni)
Collecting cdiff (from patroni)
Requirement already up-to-date: six>=1.7 in /usr/local/lib/python2.7/dist-packages (from patroni)
Collecting python-etcd<0.5,>=0.4.3 (from patroni)
Requirement already up-to-date: prettytable>=0.7 in /usr/local/lib/python2.7/dist-packages (from patroni)
Requirement already up-to-date: tzlocal in /usr/local/lib/python2.7/dist-packages (from patroni)
Requirement already up-to-date: boto in /usr/local/lib/python2.7/dist-packages (from patroni)
Collecting python-consul>=0.7.0 (from patroni)
Downloading python_consul-0.7.2-py2.py3-none-any.whl
Requirement already up-to-date: psycopg2>=2.6.1 in /usr/local/lib/python2.7/dist-packages (from patroni)
Requirement already up-to-date: kazoo==2.2.1 in /usr/local/lib/python2.7/dist-packages (from patroni)
Requirement already up-to-date: urllib3>=1.9 in /usr/local/lib/python2.7/dist-packages (from patroni)
Requirement already up-to-date: click>=4.1 in /usr/local/lib/python2.7/dist-packages (from patroni)
Requirement already up-to-date: python-dateutil in /usr/local/lib/python2.7/dist-packages (from patroni)
Collecting psutil (from patroni)
Requirement already up-to-date: PyYAML in /usr/local/lib/python2.7/dist-packages (from patroni)
Requirement already up-to-date: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests->patroni)
Requirement already up-to-date: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests->patroni)
Requirement already up-to-date: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests->patroni)
Requirement already up-to-date: dnspython>=1.13.0 in /usr/local/lib/python2.7/dist-packages (from python-etcd<0.5,>=0.4.3->patroni)
Requirement already up-to-date: pytz in /usr/local/lib/python2.7/dist-packages (from tzlocal->patroni)
Installing collected packages: cdiff, python-etcd, python-consul, psutil, patroni
Found existing installation: python-etcd 0.4.3
Uninstalling python-etcd-0.4.3:
Successfully uninstalled python-etcd-0.4.3
Found existing installation: python-consul 0.6.0
Uninstalling python-consul-0.6.0:
Successfully uninstalled python-consul-0.6.0
Found existing installation: patroni 1.0
Uninstalling patroni-1.0:
Successfully uninstalled patroni-1.0
Successfully installed cdiff-1.0 patroni-1.3.6 psutil-5.4.1 python-consul-0.7.2 python-etcd-0.4.5
You are using pip version 7.1.2, however version 9.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
I had to use sudo if I do just pip install patroni --upgrade then it is failing with the below error:-
pip install patroni --upgrade
/usr/local/lib/python2.7/dist-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
Collecting patroni
Downloading patroni-1.3.6.tar.gz (90kB)
100% |████████████████████████████████| 94kB 1.0MB/s
Requirement already up-to-date: urllib3>=1.9 in /usr/local/lib/python2.7/dist-packages (from patroni)
Requirement already up-to-date: boto in /usr/local/lib/python2.7/dist-packages (from patroni)
Requirement already up-to-date: psycopg2>=2.6.1 in /usr/local/lib/python2.7/dist-packages (from patroni)
Requirement already up-to-date: PyYAML in /usr/local/lib/python2.7/dist-packages (from patroni)
Requirement already up-to-date: requests in /usr/local/lib/python2.7/dist-packages (from patroni)
Requirement already up-to-date: six>=1.7 in /usr/local/lib/python2.7/dist-packages (from patroni)
Requirement already up-to-date: kazoo==2.2.1 in /usr/local/lib/python2.7/dist-packages (from patroni)
Collecting python-etcd<0.5,>=0.4.3 (from patroni)
Downloading python-etcd-0.4.5.tar.gz
Collecting python-consul>=0.7.0 (from patroni)
Downloading python_consul-0.7.2-py2.py3-none-any.whl
Requirement already up-to-date: click>=4.1 in /usr/local/lib/python2.7/dist-packages (from patroni)
Requirement already up-to-date: prettytable>=0.7 in /usr/local/lib/python2.7/dist-packages (from patroni)
Requirement already up-to-date: tzlocal in /usr/local/lib/python2.7/dist-packages (from patroni)
Requirement already up-to-date: python-dateutil in /usr/local/lib/python2.7/dist-packages (from patroni)
Collecting psutil (from patroni)
Downloading psutil-5.4.1.tar.gz (408kB)
100% |████████████████████████████████| 409kB 1.1MB/s
Collecting cdiff (from patroni)
Downloading cdiff-1.0.tar.gz
Requirement already up-to-date: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests->patroni)
Requirement already up-to-date: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests->patroni)
Requirement already up-to-date: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests->patroni)
Requirement already up-to-date: dnspython>=1.13.0 in /usr/local/lib/python2.7/dist-packages (from python-etcd<0.5,>=0.4.3->patroni)
Requirement already up-to-date: pytz in /usr/local/lib/python2.7/dist-packages (from tzlocal->patroni)
Building wheels for collected packages: patroni, python-etcd, psutil, cdiff
Running setup.py bdist_wheel for patroni
Stored in directory: /home/ubuntu/.cache/pip/wheels/86/5a/e3/cb4ac4ecf20bc7a2956062d3c3ad15c660a993a0cc3aa23d35
Running setup.py bdist_wheel for python-etcd
Stored in directory: /home/ubuntu/.cache/pip/wheels/d1/79/df/26facc508cdb5fefaf3d574fb634d848754a7e69d95f53f056
Running setup.py bdist_wheel for psutil
Stored in directory: /home/ubuntu/.cache/pip/wheels/05/a2/2a/2015d6af91fb5a4cc5bcdfa9699034e2f624fc9cc5acde7ab9
Running setup.py bdist_wheel for cdiff
Stored in directory: /home/ubuntu/.cache/pip/wheels/d9/67/dc/d53a3cfea638a5652d0d2054b447d67a7d2a2bdebf0f67765e
Successfully built patroni python-etcd psutil cdiff
Installing collected packages: python-etcd, python-consul, psutil, cdiff, patroni
Found existing installation: python-etcd 0.4.3
Uninstalling python-etcd-0.4.3:
Exception:
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/pip/basecommand.py", line 211, in main
status = self.run(options, args)
File "/usr/local/lib/python2.7/dist-packages/pip/commands/install.py", line 311, in run
root=options.root_path,
File "/usr/local/lib/python2.7/dist-packages/pip/req/req_set.py", line 640, in install
requirement.uninstall(auto_confirm=True)
File "/usr/local/lib/python2.7/dist-packages/pip/req/req_install.py", line 716, in uninstall
paths_to_remove.remove(auto_confirm)
File "/usr/local/lib/python2.7/dist-packages/pip/req/req_uninstall.py", line 125, in remove
renames(path, new_path)
File "/usr/local/lib/python2.7/dist-packages/pip/utils/__init__.py", line 315, in renames
shutil.move(old, new)
File "/usr/lib/python2.7/shutil.py", line 303, in move
os.unlink(src)
OSError: [Errno 13] Permission denied: '/usr/local/lib/python2.7/dist-packages/etcd/__init__.py'
/usr/local/lib/python2.7/dist-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
You are using pip version 7.1.2, however version 9.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
Can someone let me know
Why is the permission changing after upgrade?
How can I have the same permission even after the upgrade?

I'm sure the problem is in sudo. sudo runs a new shell under root user and the new shell reads configuration files from /root/ directory. It seems /root/.bash_profile has a command umask 027 in it. Change it to 022 and the problem should be fixed.

How can I completely uninstall wagtail from my system?

I ran this command:
pip uninstall wagtail
Uninstalling wagtail: Proceed (y/n)? y
Successfully uninstalled
wagtail
However when I then ran pip install wagtail i received the following message:
root#daap:/usr/lib/python2.7# pip install wagtail
Requirement already satisfied (use --upgrade to upgrade): wagtail in /usr/local/lib/python2.7/dist-packages
Requirement already satisfied (use --upgrade to upgrade): Django<1.11,>=1.8.1 in /usr/local/lib/python2.7/dist-packages (from wagtail)
Requirement already satisfied (use --upgrade to upgrade): django-modelcluster<4.0,>=3.0 in /usr/local/lib/python2.7/dist-packages (from wagtail)
Requirement already satisfied (use --upgrade to upgrade): django-taggit<1.0,>=0.20 in /usr/local/lib/python2.7/dist-packages (from wagtail)
Requirement already satisfied (use --upgrade to upgrade): django-treebeard<5.0,>=3.0 in /usr/local/lib/python2.7/dist-packages (from wagtail)
Requirement already satisfied (use --upgrade to upgrade): djangorestframework>=3.1.3 in /usr/local/lib/python2.7/dist-packages (from wagtail)
Requirement already satisfied (use --upgrade to upgrade): Pillow>=2.6.1 in /usr/local/lib/python2.7/dist-packages (from wagtail)
Requirement already satisfied (use --upgrade to upgrade): beautifulsoup4>=4.5.1 in /usr/local/lib/python2.7/dist-packages (from wagtail)
Requirement already satisfied (use --upgrade to upgrade): html5lib<1,>=0.999 in /usr/local/lib/python2.7/dist-packages (from wagtail)
Requirement already satisfied (use --upgrade to upgrade): Unidecode>=0.04.14 in /usr/local/lib/python2.7/dist-packages (from wagtail)
Requirement already satisfied (use --upgrade to upgrade): Willow<0.5,>=0.4 in /usr/local/lib/python2.7/dist-packages (from wagtail)
Requirement already satisfied (use --upgrade to upgrade): requests<3.0,>=2.11.1 in /usr/local/lib/python2.7/dist-packages (from wagtail)
Requirement already satisfied (use --upgrade to upgrade): pytz>=2015.2 in /usr/local/lib/python2.7/dist-packages (from django-modelcluster<4.0,>=3.0->wagtail)
Requirement already satisfied (use --upgrade to upgrade): six in /usr/local/lib/python2.7/dist-packages (from html5lib<1,>=0.999->wagtail)
Cleaning up...
I am having problems that wagtail is not found when I run my django app, so I would like to completely uninstall it, then install it again.
Why isn't it being removed when I run pip uninstall wagtail? and how can I completely remove it? thanks

pip caches the python package. If you do not want to use the cache, you should use --no-cache-dir.
like this.
pip --no-cache-dir install -I wagtail
see also https://pip.pypa.io/en/stable/reference/pip_install/#caching
The cache dir is also written in this URL:)

source your-venv\bin\activate
pip install wagtail, django
To uninstall, make sure you have your venv activated. Just being in the dir does not mean you are inside virtualenv.

matplotlib import error in Mac

System: OX E
Python source: python.org
I used pip to install matplotlib and here is the result I got
sudo pip install matplotlib
The directory '/Users/clarence/Library/Caches/pip/http' or its parent
directory is not owned by the current user and the cache has been
disabled. Please check the permissions and owner of that directory. If
executing pip with sudo, you may want sudo's -H flag. The directory
'/Users/clarence/Library/Caches/pip' or its parent directory is not
owned by the current user and caching wheels has been disabled. check
the permissions and owner of that directory. If executing pip with
sudo, you may want sudo's -H flag. Requirement already satisfied (use
--upgrade to upgrade): matplotlib in /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python
Requirement already satisfied (use --upgrade to upgrade): numpy>=1.5
in
/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python
(from matplotlib) Requirement already satisfied (use --upgrade to
upgrade): python-dateutil in
/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python
(from matplotlib) Requirement already satisfied (use --upgrade to
upgrade): tornado in /Library/Python/2.7/site-packages (from
matplotlib) Requirement already satisfied (use --upgrade to upgrade):
pyparsing>=1.5.6 in
/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python
(from matplotlib) Requirement already satisfied (use --upgrade to
upgrade): nose in /Library/Python/2.7/site-packages (from matplotlib)
Requirement already satisfied (use --upgrade to upgrade): certifi in
/Library/Python/2.7/site-packages (from tornado->matplotlib)
Requirement already satisfied (use --upgrade to upgrade):
backports-abc>=0.4 in /Library/Python/2.7/site-packages (from
tornado->matplotlib) Requirement already satisfied (use --upgrade to
upgrade): backports.ssl-match-hostname in
/Library/Python/2.7/site-packages (from tornado->matplotlib)
Requirement already satisfied (use --upgrade to upgrade):
singledispatch in /Library/Python/2.7/site-packages (from
tornado->matplotlib) Requirement already satisfied (use --upgrade to
upgrade): six in
/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python
(from singledispatch->tornado->matplotlib) Clarences-MacBook-Pro:~
clarence$ sudo -H pip install matplotlib Requirement already
satisfied (use --upgrade to upgrade): matplotlib in
/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python
Requirement already satisfied (use --upgrade to upgrade): numpy>=1.5
in
/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python
(from matplotlib) Requirement already satisfied (use --upgrade to
upgrade): python-dateutil in
/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python
(from matplotlib) Requirement already satisfied (use --upgrade to
upgrade): tornado in /Library/Python/2.7/site-packages (from
matplotlib) Requirement already satisfied (use --upgrade to upgrade):
pyparsing>=1.5.6 in
/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python
(from matplotlib) Requirement already satisfied (use --upgrade to
upgrade): nose in /Library/Python/2.7/site-packages (from matplotlib)
Requirement already satisfied (use --upgrade to upgrade): certifi in
/Library/Python/2.7/site-packages (from tornado->matplotlib)
Requirement already satisfied (use --upgrade to upgrade):
backports-abc>=0.4 in /Library/Python/2.7/site-packages (from
tornado->matplotlib) Requirement already satisfied (use --upgrade to
upgrade): backports.ssl-match-hostname in
/Library/Python/2.7/site-packages (from tornado->matplotlib)
Requirement already satisfied (use --upgrade to upgrade):
singledispatch in /Library/Python/2.7/site-packages (from
tornado->matplotlib) Requirement already satisfied (use --upgrade to
upgrade): six in
/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python
(from singledispatch->tornado->matplotlib)
When I go ahead the import that library in python. python says it could not find the modulus.
python3.5 -c 'import matplotlib; print(matplotlib.__version__, matplotlib.__file__)'
Traceback (most recent call last): File "", line 1, in
ImportError: No module named 'matplotlib'
How can I fix this?

You need to install those libraries for the same version of python that you are using later to execute your code.
So if you want to use Python 2.7 you have to call
sudo pip2 install matplotlib
When using python 3.5 use it in a similar way
sudo pip3 install matplotlib

import matplotlib.pyplot - fc-list hangs

I am experiencing the same problem as described in import matplotlib.pyplot hangs after updating my matplotlib version import matplotlib.pyplot as plt hangs while executing fc-list.
However, the proposed solution does not work for me; running fc-list in ~/.matplotlib takes 100% CPU and does not finish within 15 minutes.
I have the following configuration:
Mac, running OS 10.10.5 (OS X Yosemite)
Python 2.7.5
matplotlib 1.4.3 (was 1.3.1, no problems with that version)
To update my matplotlib version, I used:
sudo pip install matplotlib
If I re-run this command now, I get
Requirement already satisfied (use --upgrade to upgrade): matplotlib in /Library/Python/2.7/site-packages
Requirement already satisfied (use --upgrade to upgrade): numpy>=1.6 in /Library/Python/2.7/site-packages (from matplotlib)
Requirement already satisfied (use --upgrade to upgrade): pytz in /Library/Python/2.7/site-packages (from matplotlib)
Requirement already satisfied (use --upgrade to upgrade): pyparsing!=2.0.0,>=1.5.6 in /Library/Python/2.7/site-packages (from matplotlib)
Requirement already satisfied (use --upgrade to upgrade): python-dateutil in /Library/Python/2.7/site-packages (from matplotlib)
Requirement already satisfied (use --upgrade to upgrade): nose>=0.11.1 in /Library/Python/2.7/site-packages (from matplotlib)
Requirement already satisfied (use --upgrade to upgrade): six>=1.4 in /Library/Python/2.7/site-packages (from matplotlib)
Requirement already satisfied (use --upgrade to upgrade): mock in /Library/Python/2.7/site-packages (from matplotlib)
Requirement already satisfied (use --upgrade to upgrade): funcsigs in /Library/Python/2.7/site-packages (from mock->matplotlib)
Requirement already satisfied (use --upgrade to upgrade): pbr>=0.11 in /Library/Python/2.7/site-packages (from mock->matplotlib)
and sudo pip install --upgrade matplotlib:
Requirement already up-to-date: matplotlib in /Library/Python/2.7/site-packages
Requirement already up-to-date: numpy>=1.6 in /Library/Python/2.7/site-packages (from matplotlib)
Requirement already up-to-date: pytz in /Library/Python/2.7/site-packages (from matplotlib)
Requirement already up-to-date: pyparsing!=2.0.0,>=1.5.6 in /Library/Python/2.7/site-packages (from matplotlib)
Requirement already up-to-date: python-dateutil in /Library/Python/2.7/site-packages (from matplotlib)
Requirement already up-to-date: nose>=0.11.1 in /Library/Python/2.7/site-packages (from matplotlib)
Requirement already up-to-date: six>=1.4 in /Library/Python/2.7/site-packages (from matplotlib)
Requirement already up-to-date: mock in /Library/Python/2.7/site-packages (from matplotlib)
Requirement already up-to-date: funcsigs in /Library/Python/2.7/site-packages (from mock->matplotlib)
Requirement already up-to-date: pbr>=0.11 in /Library/Python/2.7/site-packages (from mock->matplotlib)
I have a fontList.cache file in ~/.matplotlib. This file is old (dating back to 2013, thus older than this update attempt).
Questions:
Despite the fontList.cache file, should I run fc-list, expecting it to finish at some point?
Is matplotlib expecting to find the fontList.cache file in a different location? Maybe my install/update of matplotlib messed up some path - but: matplotlib.get_cachedir() returns /Users/MyName/.matplotlib
Has the filename changed? Should I rename fontList.cache?
I do not have the /usr/X11/lindirectory anymore. However, could fc-list still enter an infinite loop as suggested?

I cannot reproduce the problem after posting the question.
I suspect that (re-)running
sudo pip install matplotlib
and/or
sudo pip install --upgrade matplotlib
while writing the question (to capture the on-screen output) fixed the previously existing issue.
fc-list now finished within 2 minutes; hough it did not create a new fontList.cache file.
I can now call import matplotlib.pyplot as plt as before.

Install module using pip within Anaconda environment

I've recently transitioned to an Anaconda environment on RHEL6, but I need a module not in the conda repository. So I install it with pip:
(gt_keys)[user#server gtkey]$ which pip
/srv/home/shared/anaconda/envs/gt_keys/bin/pip
(gt_keys)[user#server gtkey]$ pip install owslib
Downloading/unpacking owslib
Downloading OWSLib-0.8.3.tar.gz (97kB): 97kB downloaded
Running setup.py egg_info for package owslib
Requirement already satisfied (use --upgrade to upgrade): python-dateutil>=1.5 in /srv/home/shared/anaconda/lib/python2.7/site-packages (from owslib)
Requirement already satisfied (use --upgrade to upgrade): pytz in /srv/home/shared/anaconda/lib/python2.7/site-packages (from owslib)
Requirement already satisfied (use --upgrade to upgrade): pytest in /srv/home/shared/anaconda/lib/python2.7/site-packages (from owslib)
Requirement already satisfied (use --upgrade to upgrade): pytest-cov in /srv/home/shared/anaconda/lib/python2.7/site-packages (from owslib)
Requirement already satisfied (use --upgrade to upgrade): PIL in /srv/home/shared/anaconda/lib/python2.7/site-packages/PIL (from owslib)
Requirement already satisfied (use --upgrade to upgrade): cov-core>=1.6 in /srv/home/shared/anaconda/lib/python2.7/site-packages (from pytest-cov->owslib)
Requirement already satisfied (use --upgrade to upgrade): coverage>=3.4 in /srv/home/shared/anaconda/lib/python2.7/site-packages (from cov-core>=1.6->pytest-cov->owslib)
Installing collected packages: owslib
Running setup.py install for owslib
Successfully installed owslib
Cleaning up...
But it doesn't work in my environment:
(gt_keys)[user#server gtkey]$ which python
/srv/home/shared/anaconda/envs/gt_keys/bin/python
(gt_keys)[user#server gtkey]$ python -c 'import owslib'
Traceback (most recent call last):
File "<string>", line 1, in <module>
ImportError: No module named owslib
owslib only made it to the base Ananaconda install:
sh-4.1$ which python
/srv/home/shared/anaconda/bin/python
sh-4.1$ python -c 'import owslib'
What am I doing wrong?

I don't know why, but this time it worked:
pip install owslib --allow-external PIL --allow-unverified PIL
I would be interested, still, in any comments on my process. Something strange was happening that was resolved when I reconnected to the server this morning after failing last night. Must be something with the path.