Facing an HTTPSHandler error while installing python packages using pip, following is the stack trace,
--------desktop:~$ pip install Django==1.3
Traceback (most recent call last):
File "/home/env/.genv/bin/pip", line 9, in <module>
load_entry_point('pip==1.4.1', 'console_scripts', 'pip')()
File "/home/env/.genv/lib/python2.7/site-packages/pkg_resources.py", line 378, in load_entry_point
return get_distribution(dist).load_entry_point(group, name)
File "/home/env/.genv/lib/python2.7/site-packages/pkg_resources.py", line 2566, in load_entry_point
return ep.load()
File "/home/env/.genv/lib/python2.7/site-packages/pkg_resources.py", line 2260, in load
entry = __import__(self.module_name, globals(),globals(), ['__name__'])
File "/home/env/.genv/lib/python2.7/site-packages/pip/__init__.py", line 10, in <module>
from pip.util import get_installed_distributions, get_prog
File "/home/env/.genv/lib/python2.7/site-packages/pip/util.py", line 17, in <module>
from pip.vendor.distlib import version
File "/home/env/.genv/lib/python2.7/site-packages/pip/vendor/distlib/version.py", line 13, in <module>
from .compat import string_types
File "/home/env/.genv/lib/python2.7/site-packages/pip/vendor/distlib/compat.py", line 31, in <module>
from urllib2 import (Request, urlopen, URLError, HTTPError,
ImportError: cannot import name HTTPSHandler
I used to edit Modules/setup.dist file and uncomment SSL code lines and rebuilt it, with reference to following thread : http://forums.opensuse.org/english/get-technical-help-here/applications/488962-opensuse-python-openssl-2.html
OSX + homebrew users:
You can get the latest updates to the recipe:
brew reinstall python
But if you still get the issue, e.g. maybe you have upgraded your OS, then you may need to get the latest openssl first. You can check which version and where it is used from:
openssl version -a
which openssl
To get the latest openssl:
brew update
brew install openssl
brew link --overwrite --dry-run openssl # safety first.
brew link openssl --overwrite
This may issue a warning:
bash-4.3$ brew link --overwrite --dry-run openssl
Warning: Refusing to link: openssl Linking keg-only openssl means you may end up linking against the insecure, deprecated system OpenSSL while using the headers from Homebrew's openssl.
Instead, pass the full include/library paths to your compiler e.g.:
-I/usr/local/opt/openssl/include -L/usr/local/opt/openssl/lib
Side note: this warning means that for other apps, you may want to use
export LDFLAGS=-L/usr/local/opt/openssl/lib
export CPPFLAGS=-I/usr/local/opt/openssl/include
Then recompile python:
brew uninstall python
brew install python --with-brewed-openssl
or for python 3
brew uninstall python3
brew install python3 --with-brewed-openssl
You need to install the OpenSSL header files before building Python if you need SSL support. On Debian and Ubuntu, they are in a package called libssl-dev. You might need some more dependencies, as noted here.
Homebrew
This was probably caused by an upgrade to Mavericks. Here's how I fixed it.
Update OpenSSL
#make a copy of the existing library, just in case
sudo cp /usr/bin/openssl /usr/bin/openssl.apple
# update openssl
brew update
brew install openssl
brew link --force openssl
# reload terminal paths
hash -r
Reinstall Python
Python 3
brew uninstall python3
brew install python3 --with-brewed-openssl
Python 2
brew uninstall python
brew install python --with-brewed-openssl
This answer consolidates all the Stack Exchange answers and comments I found, and is based mostly on this Apple Stack Exchange answer.
I was having this problem on Mac OSX, even after confirming my PATH, etc.
Did a; pip uninstall virtualenv then install virtualenv and it seemed to works now.
At the time I had forced brew to link openssl, unlinked it and virtualenv still seems to work but maybe that's because it was originally linked when I reinstalled it.
You need to install OpenSSl before make and install Python to solve the problem.
On Centos:
yum install openssl openssl-devel -y
source
Another symptom of this problem for me was if I went into the python console of my virtualenv and did import ssl it would error out. Turns out my virtualenv wasn't using the brew version of python, just the default install on my machine. No clue why the default install suddenly stopped working, but here's how I fixed it the problem:
rmvirtualenv myvirtualenv
brew update
brew reinstall python
mkvirtualenv -p /usr/local/Cellar/python/whatever_version_number/bin/python myvirtualenv
It seems your pip requires HTTPSHandler which is part of SSL library.
OSX
On OS X you should link OpenSSL during Python installation (See: #14497).
For Python 2:
brew reinstall python --with-brewed-openssl
pip install --upgrade pip
For Python 3:
brew reinstall python3 --with-brewed-openssl
pip3 install --upgrade pip
You could have multiple Python instances together, to list them run:
brew list | grep ^python
Or list your version via ls -al /usr/local/lib/python*.
I'm using Redhat and have met the same problem.
My solution is :
install openssl and openssl-devel ---- yum install openssl openssl-devel -y
install krb5-devel ---- yum install krb5-devel
change to your python's directory and recompile it ---- make
If I didn't do the 2nd step, when I recompiled my python2.7 , the log would say " fail to build module _ssl".
In many cases this is caused by an out of date virtualenv, here's a script to regenerate your virtualenv(s): https://gist.github.com/WoLpH/fb98f7dc6ba6f05da2b8
Simply copy it to a file (downloadable link above) and execute it like this: zsh -e recreate_virtualenvs.sh <project_name>
#!/bin/zsh -e
if [ ! -d "$PROJECT_HOME" ]; then
echo 'Your $PROJECT_HOME needs to be defined'
echo 'http://virtualenvwrapper.readthedocs.org/en/latest/install.html#location-of-project-directories'
exit 1
fi
if [ "" = "$1" ]; then
echo "Usage: $0 <project_name>"
exit 1
fi
env="$1"
project_dir="$PROJECT_HOME/$1"
env_dir="$HOME/envs/$1"
function command_exists(){
type $1 2>/dev/null | grep -vq ' not found'
}
if command_exists workon; then
echo 'Getting virtualenvwrapper from environment'
# Workon exists, nothing to do :)
elif [ -x ~/bin/mount_workon ]; then
echo 'Using mount workon'
# Optional support for packaged project directories and virtualenvs using
# https://github.com/WoLpH/dotfiles/blob/master/bin/mount_workon
. ~/bin/mount_workon
mount_file "$project_dir"
mount_file "$env_dir"
elif command_exists virtualenvwrapper.sh; then
echo 'Using virtualenvwrapper'
. $(which virtualenvwrapper.sh)
fi
if ! command_exists workon; then
echo 'Virtualenvwrapper not found, please install it'
exit 1
fi
rmvirtualenv $env || true
echo "Recreating $env"
mkvirtualenv $env || true
workon "$env" || true
pip install virtualenv{,wrapper}
cd $project_dir
setvirtualenvproject
if [ -f setup.py ]; then
echo "Installing local package"
pip install -e .
fi
function install_requirements(){
# Installing requirements from given file, if it exists
if [ -f "$1" ]; then
echo "Installing requirements from $1"
pip install -r "$1"
fi
}
install_requirements requirements_test.txt
install_requirements requirements-test.txt
install_requirements requirements.txt
install_requirements test_requirements.txt
install_requirements test-requirements.txt
if [ -d docs ]; then
echo "Found docs, installing sphinx"
pip install sphinx{,-pypi-upload} py
fi
echo "Installing ipython"
pip install ipython
if [ -f tox.ini ]; then
deps=$(python -c "
parser=__import__('ConfigParser').ConfigParser();
parser.read('tox.ini');
print parser.get('testenv', 'deps').strip().replace('{toxinidir}/', '')")
echo "Found deps from tox.ini: $deps"
echo $deps | parallel -v --no-notice pip install {}
fi
if [ -f .travis.yml ]; then
echo "Found deps from travis:"
installs=$(grep 'pip install' .travis.yml | grep -v '\$' | sed -e 's/.*pip install/pip install/' | grep -v 'pip install . --use-mirrors' | sed -e 's/$/;/')
echo $installs
eval $installs
fi
deactivate
On OSX, brew kept refusing to link against its openssl with this error:
15:27 $ brew link --force openssl
Warning: Refusing to link: openssl
Linking keg-only openssl means you may end up linking against the insecure,
deprecated system OpenSSL while using the headers from Homebrew's openssl.
Instead, pass the full include/library paths to your compiler e.g.:
-I/usr/local/opt/openssl/include -L/usr/local/opt/openssl/lib
I finally was able to get it working with:
brew remove openssl
brew uninstall --force openssl
brew install openssl
export LDFLAGS=-L/usr/local/opt/openssl/lib
export CPPFLAGS=-I/usr/local/opt/openssl/include
brew remove python
brew update
brew install python
For Ubuntu
First check wheather install openssl-develop
sudo apt-get install libssl-dev
Try another way to reinstall pip
sudo apt-get install python-setuptools
sudo easy_install pip
use setuptools to install pip rather than install with source code may can solve the problem of dependency.
Just installed ssl pyopenssl
pip install pyopenssl
Related
I installed python 3.9 following the steps in this link.
sudo apt update
sudo apt install python3.9
python3.9
sudo update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.[old-version] 1
sudo update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.9 2
sudo update-alternatives --config python3
However, it's throwing an error that python3.9 not found on the 3rd point. Also, I noticed the python3.9 on installation using the 2nd point is showing Note, selecting 'postgresql-plpython3-9.5' for regex 'python3.9'.
Complete message is
Reading package lists... Done
Building dependency tree
Reading state information... Done
Note, selecting 'postgresql-plpython3-9.5' for regex 'python3.9'
The following packages were automatically installed and are no longer required:
linux-aws-headers-4.4.0-1104 linux-aws-headers-4.4.0-1105 linux-aws-headers-4.4.0-1106 linux-aws-headers-4.4.0-1107 linux-aws-headers-4.4.0-1109 linux-aws-headers-4.4.0-1110 linux-aws-headers-4.4.0-1111
linux-aws-headers-4.4.0-1112 linux-aws-headers-4.4.0-1113 linux-aws-headers-4.4.0-1114
Use 'sudo apt autoremove' to remove them.
The following NEW packages will be installed:
postgresql-plpython3-9.5
0 upgraded, 1 newly installed, 0 to remove and 56 not upgraded.
Need to get 0 B/40.6 kB of archives.
After this operation, 166 kB of additional disk space will be used.
Selecting previously unselected package postgresql-plpython3-9.5.
(Reading database ... 362651 files and directories currently installed.)
Preparing to unpack .../postgresql-plpython3-9.5_9.5.25-0ubuntu0.16.04.1_amd64.deb ...
Unpacking postgresql-plpython3-9.5 (9.5.25-0ubuntu0.16.04.1) ...
Processing triggers for postgresql-common (173ubuntu0.3) ...
Building PostgreSQL dictionaries from installed myspell/hunspell packages...
Removing obsolete dictionary files:
Setting up postgresql-plpython3-9.5 (9.5.25-0ubuntu0.16.04.1) ...
Why is it setting up postgresql-plpython3-9.5 and how can I prevent it from doing so?
The Problem:
The deadsnakes ppa is no longer available for Ubuntu Xenial. That's the reason you cannot install python3.9. See this issue. You will have to compile from source or upgrade your server to a supported version of Ubuntu.
Solution: build it yourself
If you are not able to upgrade your system, you could instead use pyenv to install any given python version as described below. For this a new version of openssl needs to be installed for python version >= 3.8
# install dependencies
apt update
apt install -y build-essential zlib1g-dev libncurses5-dev libgdbm-dev libnss3-dev libssl-dev libsqlite3-dev libreadline-dev libffi-dev curl libbz2-dev liblzma-dev git
# download and compile openssl
curl -L https://www.openssl.org/source/openssl-1.1.1s.tar.gz | (cd /usr/src; tar xz)
cd /usr/src/openssl-1.1.1s && ./config --prefix=/usr/local && make -j4 && make install
# download and configure pyenv
curl -L https://github.com/pyenv/pyenv-installer/raw/master/bin/pyenv-installer | bash
echo >> ~/.bashrc # add new-line.
echo 'export PATH="/root/.pyenv/bin:$PATH"' >> ~/.bashrc
echo 'eval "$(pyenv init -)"' >> ~/.bashrc
echo 'eval "$(pyenv virtualenv-init -)"' >> ~/.bashrc
source ~/.bashrc
# build python 3.9.16 with pyenv
CONFIGURE_OPTS="--with-openssl=/usr/local --with-openssl-rpath=auto" pyenv install 3.9.16
# build python 3.10.9 with pyenv
CONFIGURE_OPTS="--with-openssl=/usr/local --with-openssl-rpath=auto" pyenv install 3.10.9
You may have it already installed
try running $ python3 --version
to see what python version your running.
If its not installed try running $ sudo apt-get update then run $ sudo apt-get install python3.9 to install python3.9
Hopefully this will help
You will need to add the deadsnakes repo.
sudo add-apt-repository ppa:deadsnakes/ppa
After that you can follow the steps in your question.
If you still have the same issues after adding the deadsnakes repo, it probably means your running an unsupported version of linux. You might then have to install Python 3.9 from source, you can check this answer on how to do that.
I recently upgraded a machine from Ubuntu Server 16.04 LTS to 18.04 LTS using command line. As a result, the built-in Python installation is upgraded from Python 3.5 to 3.6. And it seems that all the Python packages previously installed using pip3 are no longer accessible. I plan to clean up all those packages and use conda for Python package management. My question is, what is the best practice for a clean uninstallation of those inaccessible packages in this case?
The old packages installed via pip3 were primarily located under /usr/local/lib/python3.5/ and ~/.local/lib/python3.5/. But there could be other leftover files, e.g., under /usr/local/bin/. I would like to remove all of related files that came with pip3 install.
sudo pip install installs pip packages to/usr/local/lib/<python_version>/dist-packages, and apt packages to /usr/lib/<python_version>/dist-packages. Check these directories and remove the unwanted packages.
I ended up writing a bash script to call pip3 uninstall on each previously installed package iteratively.
#!/bin/bash
pypath_cmd="PYTHONPATH=$HOME/.local/lib/python3.5/site-packages"
export $pypath_cmd
echo "Uninstalling editable packages in $PYTHONPATH"
rm -f $PYTHONPATH/*.egg-link
rm -f $PYTHONPATH/easy-install.pth
pip3 freeze --all --local | cut --delimiter="=" -f 1 | while read pkg ; do
echo $pkg: $(pip3 show $pkg | grep "Location:")
pip3 uninstall -y $pkg
done
pypath_cmd="PYTHONPATH=/usr/local/lib/python3.5/dist-packages"
export $pypath_cmd
echo "Uninstalling editable packages in $PYTHONPATH"
sudo rm -f $PYTHONPATH/*.egg-link
sudo rm -f $PYTHONPATH/easy-install.pth
pip3 freeze --all --local | cut --delimiter="=" -f 1 | while read pkg ; do
echo $pkg: $(pip3 show $pkg | grep "Location:")
sudo $pypath_cmd pip3 uninstall -y --no-cache-dir $pkg
done
(py36venv) vagrant#pvagrant-dev-vm:/vagrant/venvs$ pip3 install pep8
pip is configured with locations that require TLS/SSL, however the ssl module in Python is not available.
Collecting pep8 Could not fetch URL
https://pypi.python.org/simple/pep8/: There was a problem confirming
the ssl certificate: Can't connect to HTTPS URL because the SSL module
is not available. - skipping
Could not find a version that satisfies the requirement pep8 (from
versions: ) No matching distribution found for pep8
Background information - Trying to move to python 3.6.
Installed python3.6 using the below commands:
wget https://www.python.org/ftp/python/3.6.0/Python-3.6.0.tgz
tar -xvf Python-3.6.0.tgz
cd Python-3.6.0
./configure --enable-optimizations
make -j8 sudo
make altinstall python3.6
Created virtualenv by:
python3.6 -m venv py36venv
source py36venv/bin/activate
Tried to install pep8
(py36venv) pip3 install pep8
pip is configured with locations that require TLS/SSL, however the ssl
module in Python is not available.
Collecting pep8
Could not fetch URL https://pypi.python.org/simple/pep8/: There was a problem
confirming the ssl certificate: Can't connect to HTTPS URL because the
SSL module is not available. - skipping Could not find a version
that satisfies the requirement pep8 (from versions: ) No matching
distribution found for pep8
I followed the below steps for python3.6 installation in ubuntu 14.04 and virtualenv pip installs works fine.
Python 3.6 Installation:
sudo apt-get install python3-dev libffi-dev libssl-dev
wget https://www.python.org/ftp/python/3.6.0/Python-3.6.0.tgz
tar xvf Python-3.6.0.tgz
cd Python-3.6.0
./configure --enable-optimizations
make -j8
sudo make altinstall
python3.6
If seeing the following error --
zipimport.ZipImportError: can't decompress data; zlib not available
make: *** [altinstall] Error 1
try:
sudo apt-get install zlib1g-dev
Validation:
Create virtualenv in python3.6:
python3.6 -m venv testenv
source testenv/bin/activate
pip install pep8
using pip:
(testenv) vagrant#pvagrant-dev-vm:~$ pip install pep8
*Collecting pep8
Downloading pep8-1.7.0-py2.py3-none-any.whl (41kB)
100% |████████████████████████████████| 51kB 4.1MB/s
Installing collected packages: pep8
Successfully installed pep8-1.7.0*
(testenv) vagrant#pvagrant-dev-vm:~$ pip list
pep8 (1.7.0)
pip (9.0.1)
setuptools (28.8.0)
I stumbled upon the same issue when I tried to create a virtual environment utilising python3.6.0. Here is my solution for Mac OS X 10.12.2 (Py_minion comment was pretty close):
Setup
I created the environment by the following steps:
downloading python3.6.0
running
./configure --prefix=<some_path>`
make
make install
mkvirtualenv --python=<some_path/bin/python3.6> foo
So basically similar to: https://stackoverflow.com/a/11301911/1286093
An indication if you have the same issue as I had would be a similar line when running make
The necessary bits to build these optional modules were not found: _ssl
Solution
Install openssl
brew install openssl
brew unlink openssl && brew link openssl --force
Change Module/Setup or Module/Setup.dist
You can find those files in the directory of the downloaded Python version.
Comment in and, if necessary change, lines 209 - 211 (I had to change the SSL variable to my openssl location).
SSL=/usr/local/opt/openssl <---- THIS DEPENDS ON YOUR INSTALLATION
_ssl _ssl.c \
-DUSE_SSL -I$(SSL)/include -I$(SSL)/include/openssl \
-L$(SSL)/lib -lssl -lcrypto
Given that this was the location of openssl
Set environment variables
export CFLAGS="-I$(brew --prefix openssl)/include"
export LDFLAGS="-L$(brew --prefix openssl)/lib"
make and install again
Running
./configure --prefix=<some_path>`
make
make install
mkvirtualenv --python=<some_path/bin/python3.6> foo
again did the trick for me
Running make reported to me in the shell output:
The necessary bits to build these optional modules were not found:
_bz2 _dbm _gdbm
_sqlite3 _ssl _tkinter
To find the necessary bits, look in setup.py in detect_modules() for the module's name.
What solved the problem in my case (Linux Mint 18.1, openssl already installed) was editing the setup.py in the Python-3.6.0 folder adding there the path to where the openssl installation put the ssl.h file on my system into ( /usr/include/openssl/ssl.h ). Here the section in which I have added the line '/usr/include':
# Detect SSL support for the socket module (via _ssl)
search_for_ssl_incs_in = [
'/usr/local/ssl/include/',
'/usr/contrib/ssl/include/',
'/usr/include/'
]
ssl_incs = find_file('openssl/ssl.h', inc_dirs,
search_for_ssl_incs_in
)
I have solved this problem on Ubuntu-16.04.1.
First you need to install necessary libraries. To install open Terminal (Ctrl+Alt+T), then type;
sudo apt-get install build-essential checkinstall
sudo apt-get install libreadline-gplv2-dev libncursesw5-dev libssl-dev libsqlite3-dev tk-dev libgdbm-dev libc6-dev libbz2-dev
After that go the directory where your python file is then reconfigure and reinstall python3.6 .
cd /opt/Python3.6/
./configure
make
sudo make install
NOTE
If you installed Python3.6 via ppa, then reinstall it again;
sudo apt-get install python3.6
Now you should be able to use pip3.6
I ran into the same error when building Python 3.6.1 from source under CentOS 7.
For CentOS7, I had to first:
sudo yum install openssl-dev
Then:
./configure --enable-optimizations
make altinstall
Now pip3.6 works :-)
A complete script can be found HERE
Install Prerequisites
For RHEL/CentOS
sudo yum -y install gcc gcc-c++ zlib zlib-devel libffi-devel openssl-devel wget
For Ubuntu/Debian
sudo apt-get -y install build-essential python-dev python-setuptools python-pip
python-smbus libncursesw5-dev libgdbm-dev libc6-dev zlib1g-dev libsqlite3-dev
tk-dev libssl-dev openssl libffi-dev wget
Download Python
Modify for the version of python you want
Python Versions
cd /var/tmp
sudo wget https://www.python.org/ftp/python/3.x.x/Python-x.x.x.tgz
sudo tar xf Python-3.*.tgz
cd Python-3*
Configure/Make/Install
sudo ./configure --enable-optimizations --enable-shared --prefix=/usr/local
sudo make && make altinstall
Cleanup Shared Library & Add to Path
Stripping the shared library of debugging symbols can speed up execution when running parallel scripts.
sudo make && make altinstall
sudo strip /usr/local/lib/libpython3.7m.so.1.0
sudo echo 'export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/usr/local/lib' >> /etc/profile.d/python.sh
sudo echo 'export PATH=${PATH}:~/usr/local/bin/' >> /etc/profile.d/python.sh
sudo echo '/usr/local/lib' >> /etc/ld.so.conf
sudo ldconfig
Reference
Gist
Stack Overflow
Stack Overflow
Daniel Erikson
Unix StackExchange
TLDP
I am trying to install the GSSAPI module through pip but I receive this error that I don't know how to resolve.
Could not find main GSSAPI shared library. Please try setting
GSSAPI_MAIN_LIB yourself or setting ENABLE_SUPPORT_DETECTION to
'false'
I need this to work on Python 2.6 for LDAP3 authentication.
Summary, for the impatient
$ sudo ln -s /usr/bin/krb5-config.mit /usr/bin/krb5-config
$ sudo ln -s /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2 /usr/lib/libgssapi_krb5.so
$ sudo apt-get install python-pip libkrb5-dev
$ sudo pip install gssapi
And now the details...
I have a Debian system that uses Heimdal Kerberos. I'll take you through what I had to do to get it working for me. Hopefully, this can help someone else as well.
Problem 1 - krb5-config: command not found
setup.py for gssapi uses the krb5-config command to find the GSSAPI library to link against (see here). Because my system was installed using Heimdal instead of MIT Kerberos, the executable command has been renamed to krb5-config.mit so setup.py misses it.
$ krb5-config --libs gssapi # doesn't work
bash: krb5-config: command not found
I created a symlink to get it to work for the install:
$ sudo ln -s /usr/bin/krb5-config.mit /usr/bin/krb5-config
$ krb5-config --libs gssapi # does work
-L/usr/lib/x86_64-linux-gnu/mit-krb5 -Wl,-z,relro -lgssapi_krb5 -lkrb5 -lk5crypto -lcom_err
Problem 2 - libgssapi_krb5.so: cannot open shared object file: No such file or directory
setup.py is looking in /usr/lib for the gssapi library to link against. In Debian Jesse, most libs are now kept in /usr/lib/x86_64-linux-gnu. Again, a symlink can fix this:
$ sudo ln -s /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2 /usr/lib/libgssapi_krb5.so
Problem 3 - error: unknown type name ‘gss_key_value_set_desc’
The build fails because it does not recognize a symbol in the library. The reason for this is that it was not able to get the right header file. Silly me, I forgot to include the -dev package for krb5 headers. Fix this with apt-get:
$ sudo apt-get install libkrb5-dev
Finally - Install gssapi
Now we should be all ready to go.
$ sudo pip install gssapi
If you want to tidy up, you can remove the symlink to the krb5-config.mit command:
$ sudo rm /usr/bin/krb5-config
sudo apt install libkrb5-dev
actually installs /usr/bin/krb5-config and /usr/lib/libgssapi_krb5.so
so none of the symlinking was needed, just install libkrb5-dev and you should be good.
For me, the issue got resolved after installing the package "krb5-libs" in Centos.
Basically we need to have libgssapi_krb5.so file for installing gssapi.
I compiled Python 2.6.6 with google-perf tools (tcmalloc) library to eliminate some of the memory issues I was having with the default 2.6.5. After getting 2.6.6 going it seems to not work becuase I think having issues with the default 2.6.5 install in Ubuntu. Will none of the binaries installed from the software channel like wxPython and setuptools work properly with 2.6.6. Do these need to be recompiled? Any other suggestions to get it working smoothly. Can I still set 2.6.5 as default without changing the Path? The path looks in usr/local/bin first.
A good general rule of thumb is to NEVER use the default system installed Python for any software development beyond miscellaneous system admin scripts. This applies on all UNIXes including Linux and OS/X.
Instead, build a good Python distro that you control, with the libraries (Python and C) that you need, and install this tarball in a non-system directory such as /opt/devpy or /data/package/python or /home/python. And why mess with 2.6 when 2.7.2 is available?
And when you are building it, make sure that all of its dependencies are in its own directory tree (RPATH) and that any system dependencies (.so files) are copied into its directory tree. Here is my version. It might not work if you just run the whole shell script. I always copy and paste sections of this into a terminal window and verify that each step worked OK. Make sure your terminal properties are set to allow lots of lines of scrollback, or only paste a couple of lines at a time.
(actually, after making a few tweaks I think this may be runnable as a script, however I would recommend something like ./pybuild.sh >pylog 2>&1 so you can comb through the output and verify that everything built OK.
This was built on Ubuntu 64 bit
#!/bin/bash
shopt -s compat40
export WGET=echo
#uncomment the following if you are running for the first time
export WGET=wget
sudo apt-get -y install build-essential
sudo apt-get -y install zlib1g-dev libxml2-dev libxslt1-dev libssl-dev libncurses5-dev
sudo apt-get -y install libreadline6-dev autotools-dev autoconf automake libtool
sudo apt-get -y install libsvn-dev mercurial subversion git-core
sudo apt-get -y install libbz2-dev libgdbm-dev sqlite3 libsqlite3-dev
sudo apt-get -y install curl libcurl4-gnutls-dev
sudo apt-get -y install libevent-dev libev-dev librrd4 rrdtool
sudo apt-get -y install uuid-dev libdb4.8-dev memcached libmemcached-dev
sudo apt-get -y install libmysqlclient-dev libexpat1-dev
cd ~
$WGET 'http://code.google.com/p/google-perftools/downloads/detail?name=google-perftools-1.7.tar.gz'
$WGET http://www.python.org/ftp/python/2.7.2/Python-2.7.2.tgz
tar zxvf Python-2.7.2.tgz
cd Python-2.7.2
#following is needed if you have an old version of Mercurial installed
#export HAS_HG=not-found
# To provide a uniform build environment
unset PYTHONPATH PYTHONSTARTUP PYTHONHOME PYTHONCASEOK PYTHONIOENCODING
unset LD_RUN_PATH LD_LIBRARY_PATH LD_DEBUG LD_TRACE_LOADED_OBJECTS
unset LD_PRELOAD SHLIB_PATH LD_BIND_NOW LD_VERBOSE
## figure out whether this is a 32 bit or 64 bit system
m=`uname -m`
if [[ $m =~ .*64 ]]; then
export CC="gcc -m64"
NBITS=64
elif [[ $m =~ .*86 ]]; then
export CC="gcc -m32"
NBITS=32
else # we are confused so bail out
echo $m
exit 1
fi
# some stuff related to distro independent build
# extra_link_args = ['-Wl,-R/data1/python27/lib']
#--enable-shared and a relative
# RPATH[0] (eg LD_RUN_PATH='${ORIGIN}/../lib')
export TARG=/data1/packages/python272
export TCMALLOC_SKIP_SBRK=true
#export CFLAGS='-ltcmalloc' # Google's fast malloc
export COMMONLDFLAGS='-Wl,-rpath,\$$ORIGIN/../lib -Wl,-rpath-link,\$$ORIGIN:\$$ORIGIN/../lib:\$$ORIGIN/../../lib -Wl,-z,origin -Wl,--enable-new-dtags'
# -Wl,-dynamic-linker,$TARG/lib/ld-linux-x86-64.so.2
export LDFLAGS=$COMMONLDFLAGS
./configure --prefix=$TARG --with-dbmliborder=bdb:gdbm --enable-shared --enable-ipv6
# if you have ia32-libs installed on a 64-bit system
#export COMMONLDFLAGS="-L/lib32 -L/usr/lib32 -L`pwd`/lib32 -Wl,-rpath,$TARG/lib32 -Wl,-rpath,$TARG/usr/lib32"
make
# ignore failure to build the following since they are obsolete or deprecated
# _tkinter bsddb185 dl imageop sunaudiodev
#install it and collect any dependency libraries - not needed with RPATH
sudo mkdir -p $TARG
sudo chown `whoami`.users $TARG
make install
# collect binary libraries ##REDO THIS IF YOU ADD ANY ADDITIONAL MODULES##
function collect_binary_libs {
cd $TARG
find . -name '*.so' | sed 's/^/ldd -v /' >elffiles
echo "ldd -v bin/python" >>elffiles
chmod +x elffiles
./elffiles | sed 's/.*=> //;s/ .*//;/:$/d;s/^ *//' | sort -u | sed 's/.*/cp -L & lib/' >lddinfo
# mkdir lib
chmod +x lddinfo
./lddinfo
cd ~
}
collect_binary_libs
#set the path
cd ~
export PATH=$TARG/bin:$PATH
#installed setuptools
$WGET http://pypi.python.org/packages/2.7/s/setuptools/setuptools-0.6c11-py2.7.egg
chmod +x setuptools-0.6c11-py2.7.egg
./setuptools-0.6c11-py2.7.egg
#installed virtualenv
tar zxvf virtualenv-1.6.1.tar.gz
cd virtualenv-1.6.1
python setup.py install
cd ~
# created a base virtualenv that should work for almost all projects
# we make it relocatable in case its location in the filesystem changes.
cd ~
python virtualenv-1.6.1/virtualenv.py /data1/py27base # first make it
python virtualenv-1.6.1/virtualenv.py --relocatable /data1/py27base #then relocatabilize
# check it out
source ~/junk/bin/activate
python --version
# fill the virtualenv with useful modules
# watch out for binary builds that may have dependency problems
export LD_RUN_PATH='\$$ORIGIN:\$$ORIGIN/../lib:\$$ORIGIN/../../lib'
easy_install pip
pip install cython
pip install lxml
pip install httplib2
pip install python-memcached
pip install amqplib
pip install kombu
pip install carrot
pip install py_eventsocket
pip install haigha
# extra escaping of $ signs
export LDFLAGS='-Wl,-rpath,\$\$$ORIGIN/../lib:\$\$$ORIGIN/../../lib -Wl,-rpath-link,\$\$$ORIGIN/../lib -Wl,-z,origin -Wl,--enable-new-dtags'
# even more complex to build this one since we need some autotools and
# have to pull source from a repository
mkdir rabbitc
cd rabbitc
hg clone http://hg.rabbitmq.com/rabbitmq-codegen/
hg clone http://hg.rabbitmq.com/rabbitmq-c/
cd rabbitmq-c
autoreconf -i
make clean
./configure --prefix=/usr
make
sudo make install
cd ~
# for zeromq we get the latest source of the library
$WGET http://download.zeromq.org/zeromq-2.1.7.tar.gz
tar zxvf zeromq-2.1.7.tar.gz
cd zeromq-2.1.7
make clean
./configure --prefix=/usr
make
sudo make install
cd ~
# need less escaping of $ signs
export LDFLAGS='-Wl,-rpath,\$ORIGIN/../lib:\$ORIGIN/../../lib -Wl,-rpath-link,\$ORIGIN/../lib -Wl,-z,origin -Wl,--enable-new-dtags'
pip install pyzmq
pip install pylibrabbitmq # need to build C library and install first
pip install pylibmc
pip install pycurl
export LDFLAGS=$COMMONLDFLAGS
pip install cherrypy
pip install pyopenssl # might need some ldflags on this one?
pip install diesel
pip install eventlet
pip install fapws3
pip install gevent
pip install boto
pip install jinja2
pip install mako
pip install paste
pip install twisted
pip install flup
pip install pika
pip install pymysql
# pip install py-rrdtool # not on 64 bit???
pip install PyRRD
pip install tornado
pip install redis
# for tokyocabinet we need the latest source of the library
$WGET http://fallabs.com/tokyocabinet/tokyocabinet-1.4.47.tar.gz
tar zxvf tokyocabinet-1.4.47.tar.gz
cd tokyocabinet-1.4.47
make clean
./configure --prefix=/usr --enable-devel
make
sudo make install
cd ..
$WGET http://fallabs.com/tokyotyrant/tokyotyrant-1.1.41.tar.gz
tar zxvf tokyotyrant-1.1.41.tar.gz
cd tokyotyrant-1.1.41
make clean
./configure --prefix=/usr --enable-devel
make
sudo make install
cd ..
pip install tokyo-python
pip install solrpy
pip install pysolr
pip install sunburnt
pip install txamqp
pip install littlechef
pip install PyChef
pip install pyvb
pip install bottle
pip install werkzeug
pip install BeautifulSoup
pip install XSLTools
pip install numpy
pip install coverage
pip install pylint
# pip install PyChecker ???
pip install pycallgraph
pip install mkcode
pip install pydot
pip install sqlalchemy
pip install buzhug
pip install flask
pip install restez
pip install pytz
pip install mcdict
# need less escaping of $ signs
pip install py-interface
# pip install paramiko # pulled in by another module
pip install pexpect
# SVN interface
$WGET http://pysvn.barrys-emacs.org/source_kits/pysvn-1.7.5.tar.gz
tar zxvf pysvn-1.7.5.tar.gz
cd pysvn-1.7.5/Source
python setup.py backport
python setup.py configure
make
cd ../Tests
make
cd ../Sources
mkdir -p $TARG/lib/python2.7/site-packages/pysvn
cp pysvn/__init__.py $TARG/lib/python2.7/site-packages/pysvn
cp pysvn/_pysvn_2_7.so $TARG/lib/python2.7/site-packages/pysvn
cd ~
# pip install protobuf #we have to do this the hard way
$WGET http://protobuf.googlecode.com/files/protobuf-2.4.1.zip
unzip protobuf-2.4.1.zip
cd protobuf-2.4.1
make clean
./configure --prefix=/usr
make
sudo make install
cd python
python setup.py install
cd ~
pip install riak
pip install ptrace
pip install html5lib
pip install metrics
#redo the "install binary libraries" step
collect_binary_libs
# link binaries in the lib directory to avoid search path errors and also
# to reduce the number of false starts to find the library
for i in `ls $TARG/lib/python2.7/lib-dynload/*.so`
do
ln -f $i $TARG/lib/`basename $i`
done
# for the same reason link the whole lib directory to some other places in the tree
ln -s ../.. $TARG/lib/python2.7/site-packages/lib
# bundle it up and save it for packaging
cd /
tar cvf - .$TARG |gzip >~/py272-$NBITS.tar.gz
cd ~
# after untarring on another machine, we have a program call imports.py which imports
# every library as a quick check that it works. For a more positive check, run it like this
# strace -e trace=stat,fstat,open python imports.py >strace.txt 2>&1
# grep -v ' = -1' strace.txt |grep 'open(' >opens.txt
# sed <opens.txt 's/^open("//;s/".*//' |sort -u |grep -v 'dynload' |grep '\.so' >straced.txt
# ls -1d /data1/packages/python272/lib/* |sort -u >lib.txt
# then examine the strace output to see how many places it searches before finding it.
# a successful library load will be a call to open that doesn't end with ' = -1'
# If it takes too many tries to find a particular library, then another symbolic link may
# be a good idea
I'm pretty sure you have to compile wxPython to the version of Python that you want to use it with. That's always been the case with anyone else who has done something like this on the wxPython mailing list. I think that applies to most packages and especially so if they have any C/C++ components, like wxPython does. Pure Python packages can sometimes be transferred from one version to the next intact in my experience.
There are fairly extensive wxPython build instructions here: http://wxpython.org/BUILD-2.8.html
Robin Dunn and others on the wxPython mailing list are very helpful if you run into any problems.
If you compiled 2.6.6 and installed 2.6.5 from the repos, then ubuntu is having a conflict in finding what python you're using.
I'm flagging this to move to Superuser.