Directing Python to look in another folder for modules - python

I'm still pretty new to python so forgive me if this is extremely simple or extremely the wrong way of thinking about this.
I have python 2.7 installed. From what I understand when I run the following code, it lists the directories where it looks for modules.
Python 2.7.12 (default, Oct 11 2016, 14:42:58)
[GCC 4.2.1 Compatible Apple LLVM 7.3.0 (clang-703.0.31)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> print '\n'.join(sys.path)
/usr/local/lib/python2.7/site-packages
/usr/lib/python2.7/site-packages
/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python27.zip
/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7
/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-darwin
/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac
/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac/lib-scriptpackages
/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-tk
/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-old
/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/readline
/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-dynload
/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages
I have another directory that appears to have a bunch of python modules I installed in it. "/Library/Python/2.7/site-packages"
I guess I need to do one of two things:
(1) direct python to look in this additional folder for modules. How do I do that?
(2) install the modules in one of the folders it is already directing to. I've been using pip to install modules, and I think pip is installing to this additional director. How do I check if pip is installing to this folder? How do I change where pip install packages to?
Thanks!

This stuff is setup by site.py. You can find out which site.py by entering the interpreter with:
python -v
Alternatively, import site in the interactive interpreter and check the site.__file__ attribute.
There's also a helpful script in site.py which you can run with
python -m site
What you want to see in the output there is your user site, something like
USER_BASE: '/home/<your_username>/.local' (exists)
USER_SITE: '/home/<your_username>/.local/lib/python2.7/site-packages' (exists)
ENABLE_USER_SITE: True
When you pip install a package, use pip install --user, which will install modules to your user space. Don't install stuff with sudo pip install. Don't munge the sys.path manually like the "quick fix" from jmd_dk suggests. Do it properly.
If, after reading the site documentation and PEP370, you are still having trouble setting up the user site correctly, then it is acceptable to add a line like this in your bash profile:
export PYTHONPATH=/home/<your_username>/.local/lib/python2.7/site-packages
If you have both a Python 3 and a Python 2 installation, beware that the PYTHONPATH environment variable will be seen by both interpreters. In this case, it is strongly recommend to enable the site.USER_SITE separately for each interpreter (or use virtual environments) to provide adequate namespacing of installed packages.

Related

cannot use pyperclip module in python shell after successful download [duplicate]

During the past years, I have installed many Python libraries with various Python versions. To make them ready to work immediately, I installed them blindly without control. Currently they're causing problems when I tried to install pynest which invokes numpy, scipy and matplotlib. After struggling, I am going to clean and reinstall Python and the libraries.
After investigation, I found Python 2.5/2.6/2.7/3.2 on my system, and each of them has some copies or other things at: (my OS == Mac OS X 10.7.5 Lion)
/Library/Frameworks/
/opt/local/Library/Frameworks/
/opt/local/bin/
/Applications/
/usr/local/bin/
/usr/bin/
/System/Library/Frameworks/
I know I'm crazy to have these. Now I have removed all these except the things in /System/Libarary/Frameworks (I never remove any thing from /System/Library/). After the clean work, which python now gives /usr/bin/python which links to /System/Library/Frameworks.
Now, is it a clear environment for me to reinstall python? How to double check that there's no other versions existing? How should I reinstall them to guarantee that they and their libraries won't be everywhere and have many copies again?
I want to install a clean Python 2.7 onto a proper location, and make my system know exactly where it is and never install any libraries somewhere else. Please give me some advice that how to manage it like in a professional way.
For your information, here is my current $PATH, I think it should be modified:
/opt/local/bin:/opt/local/sbin:/opt/nest/lib/python2.7/site-packages:/usr/local/lib/python2.7/site-packages:/Library/Frameworks/Python.framework/Versions/2.7/bin:/usr/texbin:/Library/Frameworks/Python.framework/Versions/3.2/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/usr/texbin:/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packages/django/bin:/usr/X11/bin:/opt/local/bin:/opt/local/sbin:/usr/local/lib/python2.7/site-packages:/Library/Frameworks/Python.framework/Versions/2.7/bin:/Library/Frameworks/Python.framework/Versions/3.2/bin
Please let me know If you need more information. Thank you!
UPDATE:
I'm rethinking profoudly why it becomes so crazy. I believe it's because I installed things via:
easy_install / macports / homebrew / fink / pip sometimes;
.dmg sometimes;
.pkg sometimes;
compile source code sometimes;
and they made things at different locations. I wonder what's the mechanism behind these ways? How do they choose target location? How to prevent them from messing things up?
Why did it get messed up?
There're a couples of different way to install Python, as the update of OP says, and they locate files in different locations. For example, macports puts things into /opt/local/, while homebrew puts things into /usr/local/. Also, Mac OS X brings a few python versions with itself. So, if you install python many times via different ways, you will get many python versions existing independently on your system.
What problem does it cause?
I don't know exactly. I guess the problem is that if you have many versions of python, then which one to use and where to find packages will be determined by the path order in your system PATH and the PYTHONPATH respectively. So you may lose control of where to install python modules. Consider that if you run sudo python setup.py install to install a module (it finds python by the root's PATH) and then try to import the module by python -c "import it" (this time it finds python by your PATH), maybe something will go wrong. This is my guess, I didn't validate it. But in my own case, something did go wrong.
How to avoid this?
I think the principle would be that be aware of that different ways and tools install things independently to different locations, so use them mindfully.
Unless you intend to, don't install the same thing twice via different
ways. (If you intend to do it for python, you might want to check out virtualenv)
Keep an eye on the path order in your PATH and consider if it's
correct.
When installing modules, be clear which python (or pip) is
running and where the module is installed.
So, how did I solve my own case?
Since it had been messing up already and seemed to be very hard to cure, so finally I solved this question by a full OS re-installation, and started to follow the DOs-and-DONTs above. For the installation of the scientific environment with python (numpy/scipy/matplotlib, which had shown problems to make me ask this question), I found this tutorial was extremely helpful. So, problem solved finally.
Here is what was confusing me and how I solved it.
$ which python
/usr/bin/python
$ which python3
/usr/local/bin/python3
$ ls /usr/local/bin/python
ls: /usr/local/bin/python: No such file or directory
So notice I didn't have a HomeBrew installation of python2.7, but did have the python3 installation. The version under /usr/bin/python is using the system default. You can tell based on the module search path:
$ /usr/bin/python
Python 2.7.10 (default, Feb 7 2017, 00:08:15)
[GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.34)] on darwin
`enter code here`Type "help", "copyright", "credits" or "license" for
more information.
>>> import sys
>>> sys.path
['', '/Library/Python/2.7/...
Notice the '/Library/Python'... that's Mac OS's version of python. But I want to stay strictly on a user installed version (i.e. HomeBrew).
So here's what I did to fix this:
$ brew install python
...
Warning: python 2.7.13 is already installed, it's just not linked.
You can use `brew link python` to link this version.
$ brew link --overwrite python
$ which python
/usr/local/bin/python
$ python
Python 2.7.10 (default, Feb 7 2017, 00:08:15)
[GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.34)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.path
['', '/usr/local/Cellar/python/2.7.13...
Its no longer /Library/.. but /usr/local.
Now its finding all of my pip installed modules! Problem solved!
UPDATE:
After updating brew to version 1.5.4, it seems the symbolic links were removed. And now you have to add this to your path:
export PATH="/usr/local/opt/python/libexec/bin:$PATH"
Read the Caveats section in 'brew info python':
==> Caveats
This formula installs a python2 executable to /usr/local/bin.
If you wish to have this formula's python executable in your PATH then add
the following to ~/.bash_profile:
export PATH="/usr/local/opt/python/libexec/bin:$PATH"
Pip and setuptools have been installed. To update them
pip2 install --upgrade pip setuptools
You can install Python packages with
pip2 install <package>
They will install into the site-package directory
/usr/local/lib/python2.7/site-packages
See: https://docs.brew.sh/Homebrew-and-Python.html
In order to install a python distributions into specific folder, you can use the --prefix scheme during python installation. Using the prefix scheme, you can for example install Python 2.7 into the folder /opt/py27. Now, in order to use the new installed Python distribution you have to: cleanup you PATH and LD_LIBRARY_PATH:
Remove all 'old' Python paths and
configure (according to my example) the environment variables like this:
PATH: Add /opt/py27/bin
LD_LIBRARY_PATH: Add /opt/py27/lib
That's it.
(In case you need multiple environments of Python installed at the same time, I'd suggest to have a look at virtualenv)
tl;dr
brew install python
Symptoms
I had similar issues with python programs not finding dependencies.
My python3 version was a broken symlink.
My pip was pointing to a python 3.8
And my pip3 was pointing to 3.9
python -V was outputting some python 2.7 version
python3 -V was outputting some python3.8 version
Solution
I ran brew install python and it fixed all my problems.

pip installed module but python gives Import error [duplicate]

During the past years, I have installed many Python libraries with various Python versions. To make them ready to work immediately, I installed them blindly without control. Currently they're causing problems when I tried to install pynest which invokes numpy, scipy and matplotlib. After struggling, I am going to clean and reinstall Python and the libraries.
After investigation, I found Python 2.5/2.6/2.7/3.2 on my system, and each of them has some copies or other things at: (my OS == Mac OS X 10.7.5 Lion)
/Library/Frameworks/
/opt/local/Library/Frameworks/
/opt/local/bin/
/Applications/
/usr/local/bin/
/usr/bin/
/System/Library/Frameworks/
I know I'm crazy to have these. Now I have removed all these except the things in /System/Libarary/Frameworks (I never remove any thing from /System/Library/). After the clean work, which python now gives /usr/bin/python which links to /System/Library/Frameworks.
Now, is it a clear environment for me to reinstall python? How to double check that there's no other versions existing? How should I reinstall them to guarantee that they and their libraries won't be everywhere and have many copies again?
I want to install a clean Python 2.7 onto a proper location, and make my system know exactly where it is and never install any libraries somewhere else. Please give me some advice that how to manage it like in a professional way.
For your information, here is my current $PATH, I think it should be modified:
/opt/local/bin:/opt/local/sbin:/opt/nest/lib/python2.7/site-packages:/usr/local/lib/python2.7/site-packages:/Library/Frameworks/Python.framework/Versions/2.7/bin:/usr/texbin:/Library/Frameworks/Python.framework/Versions/3.2/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/usr/texbin:/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packages/django/bin:/usr/X11/bin:/opt/local/bin:/opt/local/sbin:/usr/local/lib/python2.7/site-packages:/Library/Frameworks/Python.framework/Versions/2.7/bin:/Library/Frameworks/Python.framework/Versions/3.2/bin
Please let me know If you need more information. Thank you!
UPDATE:
I'm rethinking profoudly why it becomes so crazy. I believe it's because I installed things via:
easy_install / macports / homebrew / fink / pip sometimes;
.dmg sometimes;
.pkg sometimes;
compile source code sometimes;
and they made things at different locations. I wonder what's the mechanism behind these ways? How do they choose target location? How to prevent them from messing things up?
Why did it get messed up?
There're a couples of different way to install Python, as the update of OP says, and they locate files in different locations. For example, macports puts things into /opt/local/, while homebrew puts things into /usr/local/. Also, Mac OS X brings a few python versions with itself. So, if you install python many times via different ways, you will get many python versions existing independently on your system.
What problem does it cause?
I don't know exactly. I guess the problem is that if you have many versions of python, then which one to use and where to find packages will be determined by the path order in your system PATH and the PYTHONPATH respectively. So you may lose control of where to install python modules. Consider that if you run sudo python setup.py install to install a module (it finds python by the root's PATH) and then try to import the module by python -c "import it" (this time it finds python by your PATH), maybe something will go wrong. This is my guess, I didn't validate it. But in my own case, something did go wrong.
How to avoid this?
I think the principle would be that be aware of that different ways and tools install things independently to different locations, so use them mindfully.
Unless you intend to, don't install the same thing twice via different
ways. (If you intend to do it for python, you might want to check out virtualenv)
Keep an eye on the path order in your PATH and consider if it's
correct.
When installing modules, be clear which python (or pip) is
running and where the module is installed.
So, how did I solve my own case?
Since it had been messing up already and seemed to be very hard to cure, so finally I solved this question by a full OS re-installation, and started to follow the DOs-and-DONTs above. For the installation of the scientific environment with python (numpy/scipy/matplotlib, which had shown problems to make me ask this question), I found this tutorial was extremely helpful. So, problem solved finally.
Here is what was confusing me and how I solved it.
$ which python
/usr/bin/python
$ which python3
/usr/local/bin/python3
$ ls /usr/local/bin/python
ls: /usr/local/bin/python: No such file or directory
So notice I didn't have a HomeBrew installation of python2.7, but did have the python3 installation. The version under /usr/bin/python is using the system default. You can tell based on the module search path:
$ /usr/bin/python
Python 2.7.10 (default, Feb 7 2017, 00:08:15)
[GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.34)] on darwin
`enter code here`Type "help", "copyright", "credits" or "license" for
more information.
>>> import sys
>>> sys.path
['', '/Library/Python/2.7/...
Notice the '/Library/Python'... that's Mac OS's version of python. But I want to stay strictly on a user installed version (i.e. HomeBrew).
So here's what I did to fix this:
$ brew install python
...
Warning: python 2.7.13 is already installed, it's just not linked.
You can use `brew link python` to link this version.
$ brew link --overwrite python
$ which python
/usr/local/bin/python
$ python
Python 2.7.10 (default, Feb 7 2017, 00:08:15)
[GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.34)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.path
['', '/usr/local/Cellar/python/2.7.13...
Its no longer /Library/.. but /usr/local.
Now its finding all of my pip installed modules! Problem solved!
UPDATE:
After updating brew to version 1.5.4, it seems the symbolic links were removed. And now you have to add this to your path:
export PATH="/usr/local/opt/python/libexec/bin:$PATH"
Read the Caveats section in 'brew info python':
==> Caveats
This formula installs a python2 executable to /usr/local/bin.
If you wish to have this formula's python executable in your PATH then add
the following to ~/.bash_profile:
export PATH="/usr/local/opt/python/libexec/bin:$PATH"
Pip and setuptools have been installed. To update them
pip2 install --upgrade pip setuptools
You can install Python packages with
pip2 install <package>
They will install into the site-package directory
/usr/local/lib/python2.7/site-packages
See: https://docs.brew.sh/Homebrew-and-Python.html
In order to install a python distributions into specific folder, you can use the --prefix scheme during python installation. Using the prefix scheme, you can for example install Python 2.7 into the folder /opt/py27. Now, in order to use the new installed Python distribution you have to: cleanup you PATH and LD_LIBRARY_PATH:
Remove all 'old' Python paths and
configure (according to my example) the environment variables like this:
PATH: Add /opt/py27/bin
LD_LIBRARY_PATH: Add /opt/py27/lib
That's it.
(In case you need multiple environments of Python installed at the same time, I'd suggest to have a look at virtualenv)
tl;dr
brew install python
Symptoms
I had similar issues with python programs not finding dependencies.
My python3 version was a broken symlink.
My pip was pointing to a python 3.8
And my pip3 was pointing to 3.9
python -V was outputting some python 2.7 version
python3 -V was outputting some python3.8 version
Solution
I ran brew install python and it fixed all my problems.

Anaconda Runtime Error: Python is not installed as a framework?

I've installed Anaconda with the pkg installer:
Python 2.7.10 |Continuum Analytics, Inc.| (default, May 28 2015, 17:04:42)
[GCC 4.2.1 (Apple Inc. build 5577)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
Anaconda is brought to you by Continuum Analytics.
Please check out: http://continuum.io/thanks and https://binstar.org
but when I attempt to use anything from matplotlib, i.e.:
from matplotlib import pyplot as plt
I get
RuntimeError: Python is not installed as a framework.
The Mac OS X backend will not be able to function correctly if Python is not installed
as a framework. See the Python documentation for more information on installing Python
as a framework on Mac OS X. Please either reinstall Python as a framework,
or try one of the other backends.
I'm really not sure what this means, or how to go about fixing it.
Posting since I just had this issue and this was a quick fix:
If you used pip to install:
Create ~/.matplotlib/matplotlibrc
Add "backend: TkAgg" (without the quotations) to the file.
I was having the same problem with anaconda 2 & matplotlib 1.5.3.
Running a simple conda install matplotlib to reinstall matplotlib did the trick for me.
If the problem is only matplotlib, is worth try to change the backend:
import matplotlib
matplotlib.use('TkAgg')
import matplotlib.pyplot as plt
plt.plot([1, 2, 3], [4, 5, 6])
plt.show()
If it works you can change the backend permanently from the matplotlibrc file.
I was having the same problem. Installing an older version of matplotlib did the trick for me. Try this command in your terminal while in your virtual environment:
pip install matplotlib==1.4.3
Run the file using pythonw instead of python.
This happens because python is not installed as a framework.
Therefore use pythonw myScript.py instead of python myScript.py
I am sure this will fix it.
I had a similar error.
RuntimeError: Python is not installed as a framework. The Mac OS X backend will not be able to function correctly if Python is not installed as a framework. See the Python documentation for more information on installing Python as a framework on Mac OS X. Please either reinstall Python as a framework, or try one of the other backends. If you are using (Ana)Conda please install python.app and replace the use of 'python' with 'pythonw'. See 'Working with Matplotlib on OSX' in the Matplotlib FAQ for more information.
From the matplotlib documentation;
$ conda install python.app
You need a framwork build of Python for matplotlib, but
The default python provided in (Ana)conda is not a framework build. However, a framework build can easily be installed, both in the main environment and in conda envs: install python.app (conda install python.app) and use pythonw rather than python
NB I had to add the conda-forge channel as python.app isn't included in the default miniconda channels
$ conda config --add channels conda-forge
If you experience this error, don't forget to check your bash_profile.
You can do this in terminal by:
cd
then
nano .bash_profile
check the contents. Macports and Homebrew add their own headings for things they've done here. You can remove the declarations they make to $PATH. Just leave the one Anaconda has made. I had a If you would like, you can:
cp .bash_profile ./bash_profile_backup_yyyy_mm_dd
and have a backup of the file, with filename indexing to the date you changed it. That is, provided you actually put in the date in instead of just the formatting characters I'm suggesting.
source ~/.bash_profile
will refresh your system's reference to the bash_profile and you should be good to go in importing and using matplotlib
if using inside a virtualenv, I recommend following the instructions here:
http://matplotlib.org/faq/virtualenv_faq.html
A reinstall of matplotlib should fix the issue for you as it did for me with
conda install matplotlib
Quickfix: Run your file using pythonw, instead of python.
e.g pythonw testFile.py.

How to check whether I have IPython installed on my machine and what sequence to install these libraries?

I need to use python for a machine learning course and I also need to install some external libraries. I'm a bit confused as to what the correct order is for installation of everything as I've heard the paths can get messed up if done incorrectly.
Here is what I need:
Python version 2.7
IPython
The libraries available inside this package by enthought
So 1st step is to see whether I have python installed
Yes: I have this version
AM#~ >python
Python 2.7.2 (default, Jun 20 2012, 16:23:33)
[GCC 4.2.1 Compatible Apple Clang 4.0 (tags/Apple/clang-418.0.60)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
Next IPython
How do I check if I have IPython?
Also whats the ideal procedure to install it? Any caveats? Any special path settings?
Can I use brew to install it?
Enthought libraries
Do I have to install these libraries in any particular order? before IPython? after IPython?
Do I have to set any specific paths?
Im trying to avoid having to install and make an error and then reinstall etc.
So any help would be much appreciated. Im running Mac OS X 10.7 (Mountain Lion).
Thanks
You can simply check the version, as we do with other software packages to check if a software package is installed properly or not.
Type in your terminal.
ipython --version
It'll print the version if ipython is installed properly.
It is so simple to get all IPython, Matplotlib, Scipy etc. now that you have Python 2.7 installed. If this is for a course, I assume you have a .edu email address with this institution? If so just go here: http://www.enthought.com/products/edudownload.php
Submit your info. Download and install it like any other program. You will then be able to call ipython from the terminal. Or to use matplotlib call ipython --pylab from the terminal.
Definitely do not go about installing the libraries one-by-one unless you are very familiar with how that works.

Can python.org releases play nice with Apple framework builds?

(Advanced apologies for the lack or proper links; the system won't allow me to add more than two.)
Unfortunately, I've learnt the hard way that you shouldn't mess with the default Python installations in Mac OS X (specifically, 10.6.8).
After using the python.org installers for 2.6.6 (http://www.python.org/getit/releases/2.6.6/) and 2.5.4 (http://www.python.org/getit/releases/2.5.4/), I have Python versions which are more mature than those provided by Apple (which is great for development), but have broken core system functionality (which is bad for just about everything else.) The most visible breaks so far have been when trying to run namebench (https://code.google.com/p/namebench/), Blink (http://icanblink.com/) and Mercurial (https://www.mercurial-scm.org/). From what I can gather, it's down to the paths.
In a default installation, the paths should resemble something those outlined in this question. Instead, mine look like this:
$ /usr/bin/python2.6 -V
Python 2.6.6
$ which python
/usr/bin/python
$ python
Python 2.6.6 (r266:84374, Aug 31 2010, 11:00:51)
[GCC 4.0.1 (Apple Inc. build 5493)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> for i in sys.path:
... print i
...
/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/setuptools-0.6c11-py2.6.egg
/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/pip-1.0.2-py2.6.egg
/Library/Frameworks/Python.framework/Versions/2.6/lib/python26.zip
/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6
/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/plat-darwin
/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/plat-mac
/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/plat-mac/lib-scriptpackages
/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/lib-tk
/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/lib-old
/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/lib-dynload
/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages
(It's a similar story for 2.5 but, for simplicity, I'll stick with 2.6.)
The issue seems to be the non-inclusion of:
/Library/Python/2.6/site-packages/*.egg
/Library/Python/2.6/site-packages
/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6
/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python
This explains why Blink and namebench can't find PyObjC (Is PyObjC pre-installed on OSX SL?) and Mercurial can't find its modules (in /Library/Python/2.6)
Issue #4865 in the Python bug tracker partially addresses this particular problem, which has since been fixed for versions 2.7 and greater. But because it's "a feature request and not a bugfix", has not been back-ported to 2.5 and 2.6. Even then, looking at the committed fix (http://hg.python.org/lookup/r70778), I'm not sure it addresses the lack of references to the "Extras" directories.
I understand that I can manually add paths to Python by manually altering site.py. I could also make use of the PYTHONPATH environment variable. I'd rather not cause any further damage by altering site.py, and changes PYTHONPATH would only be valid for scripts/applications run from the shell using my user account.
Can I get these more recent versions of Python to reference the same paths as the default framework installations? If so, what is the best way to go about it? If not, is there an accepted method of rolling back to the system defaults?
You misunderstand how Python installations work on OS X. Each Python instance has its own site-packages directory. The standard location for framework installers is within the framework at ./lib/pythonx.y/site-packages. So for the python.org installers which install into /Library/Frameworks/Python.framework, you will find its 2.6 site-packages here:
/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/
Apple makes some modifications to the versions of Python that it ships with OS X releases. From OS X 10.5 on, the system Pythons are installed at:
/System/Library/Frameworks/Python.framework
and Apple chooses to include some extra 3rd-party packages in the non-standard ./Extras directory within each version. It also uses a non-standard location for each version's site-packages directory. They are created in /Library/Python/, presumably so that user-installed packages do not modify anything in /System/Library. So for the Apple-supplied Python 2.6, its site-packages directory is:
/Library/Python/2.6/site-packages
and can be thought of as having been extended by the packages in ./Extras.
Each Python instance has a separate site-packages directory. It is not intended and often not possible to share packages across site-packages of different instances even of the same minor version of Python, i.e. 2.6. The most obvious problem is that there are often differences in the C compiler version, the OS X ABI (MACOSX_DEPLOYMENT_TARGET), the SDK version, and/or the CPU architectures used to build the Python interpreters and which are subsequently used by Distutils to build C extension modules included in 3rd-party packages.
In Mac OS X 10.6, the Apple-supplied Python is built using gcc-4.2 and targeted just for OSX 10.6 and includes 3 CPU architectures (i386, x86_64, and ppc). The python.org installers for Python 2.6 are built to run on older systems as well, so have a target of 10.3 and later, use gcc-4.0, and are 32-bit only (i386 and ppc). So, in general, you cannot run C extension modules built for the one Python with the other.
This means that, in general, you need to install separate copies of 3rd-party packages you need with and for each Python, if they are not already included with that Python. This includes basic items like easy_install (from setuptools or Distribute). The system Pythons in 10.5+ include easy_install version in /usr/bin for them. If you install a python.org Python, you'll need to install a separate version for it; by default, the easy_install command will be installed in that Python's ./bin directory in its framework; that is the Distutils default location. That's why it is recommended that you add this directory to your shell PATH (and the python.org installer for Python 2 do that automatically by default).
The change introduced by Issue4865 is not really a good solution and may fail with C extension modules. I would not depend on it remaining in Python in future versions.
Also installing a python.org Python in no way breaks anything in the system Python because they are completely independent installations using different file system locations. The only thing that may change is which instance of Python is invoked when you type a particular name. That is controlled primarily by the search order of your shell PATH environment variable. As noted, the python.org installer by default changes that order but the system Python is still readily available by using its absolute path /usr/bin/python2.6. Or you can revert the changes to shell profile, for instance, .bash_profile.
$ echo $PATH
/Library/Frameworks/Python.framework/Versions/2.6/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/usr/X11/bin
$ which python python2.6
/Library/Frameworks/Python.framework/Versions/2.6/bin/python
/Library/Frameworks/Python.framework/Versions/2.6/bin/python2.6
$ python -V
Python 2.6.6
$ python2.6 -V
Python 2.6.6
$ /usr/bin/python -V
Python 2.6.1
$ /usr/bin/python2.6 -V
Python 2.6.1
#
# remove python.org Python 2.6 from PATH
#
$ export PATH=/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/usr/X11/bin
$ which python python2.6
/usr/bin/python
/usr/bin/python2.6
$ python -V
Python 2.6.1
$ python2.6 -V
Python 2.6.1

Categories

Resources