Develop Reference

Python interpteter can't find module in virtualenv, but pip sees it and won't install

I'm trying to use the pytools module within the virtualenv created by Nervana for their Neon deep learning package, but can't seem to either find pytools or pip it. When I enter my virtualenv, I see this behavior:
me#ARL--M6800:~/Downloads/neon$ source .venv/bin/activate
(.venv) me#ARL--M6800:~/Downloads/neon$ python
Python 2.7.6 (default, Jun 22 2015, 17:58:13)
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import pytools
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named pytools
>>>
>>> import sys
>>> sys.path
['', '/usr/local/lib/python2.7/dist-packages', '/home/me/Downloads/neon',
'/home/me/Downloads/neon/.venv/lib/python2.7',
'/home/me/Downloads/neon/.venv/lib/python2.7/plat-x86_64-linux-gnu',
'/home/me/Downloads/neon/.venv/lib/python2.7/lib-tk',
'/home/me/Downloads/neon/.venv/lib/python2.7/lib-old',
'/home/me/Downloads/neon/.venv/lib/python2.7/lib-dynload',
'/usr/lib/python2.7',
'/usr/lib/python2.7/plat-x86_64-linux-gnu',
'/usr/lib/python2.7/lib-tk',
'/home/me/Downloads/neon/.venv/local/lib/python2.7/site-packages',
'/home/me/Downloads/neon/.venv/lib/python2.7/site-packages']
(.venv) me#ARL--M6800:~/Downloads/neon$ pip install pytools
Requirement already satisfied (use --upgrade to upgrade): pytools in
/usr/local/lib/python2.7/dist-packages/pytools-2016.1-py2.7.egg
Requirement already satisfied (use --upgrade to upgrade):
decorator>=3.2.0 in /usr/local/lib/python2.7/dist-packages (from pytools)
Requirement already satisfied (use --upgrade to upgrade): appdirs>=1.4.0
in /usr/local/lib/python2.7/dist-packages/appdirs-1.4.0-py2.7.egg (from
pytools)
Requirement already satisfied (use --upgrade to upgrade): six>=1.8.0 in
/usr/local/lib/python2.7/dist-packages (from pytools)
Requirement already satisfied (use --upgrade to upgrade): numpy>=1.6.0 in
/usr/local/lib/python2.7/dist-packages (from pytools)
So, I can't import pytools becauseit isn't on my sys.path. According to pip, it is installed in the /usr/local/lib/python2.7/dist-packages/pytools-2016.1-py2.7.egg directory, which leaves me with 3 questions:
First: Why can my virtualenv see my system-wide packages? I thought the default was not to see them. When I look at the Makefile used to create the virtualenv, I see this
# where our installed python packages will live
VIRTUALENV_DIR := .venv
VIRTUALENV_EXE := virtualenv -p python2.7 # use pyvenv for python3 install
ACTIVATE := $(VIRTUALENV_DIR)/bin/activate
which should give me default behavior.
Second: Why are there egg directories in my dist-packages dir? Doesn't this make it harder to find those modules? (Though apparently, the sys.path for my system environment python has been updated to search in the egg dir. When/How???)
Third: What is an efficient way of fixing things so that my virtualenv will have access to pytools?
(I would've numbered my list instead of First/Second/Third, but then the Makefile code I inserted didn't format well)

First of all, I don't know anything about Nervana's Neon.
I get the same sys.path when the virtualenv is created with root privileges:
dm#Z580:~$ sudo virtualenv test1 -p python2.7
[sudo] password for dm:
Running virtualenv with interpreter /usr/bin/python2.7
New python executable in test1/bin/python2.7
Also creating executable in test1/bin/python
Installing setuptools, pip, wheel...done.
dm#Z580:~$ source test1/bin/activate
(test1)dm#Z580:~$ which python
/home/dm/test1/bin/python
(test1)dm#Z580:~$ python
Python 2.7.6 (default, Jun 22 2015, 17:58:13)
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> import pprint
>>> pprint.pprint(sys.path)
['',
'/home/dm/test1/lib/python2.7',
'/home/dm/test1/lib/python2.7/plat-x86_64-linux-gnu',
'/home/dm/test1/lib/python2.7/lib-tk',
'/home/dm/test1/lib/python2.7/lib-old',
'/home/dm/test1/lib/python2.7/lib-dynload',
'/usr/lib/python2.7',
'/usr/lib/python2.7/plat-x86_64-linux-gnu',
'/usr/lib/python2.7/lib-tk',
'/home/dm/test1/local/lib/python2.7/site-packages',
'/home/dm/test1/lib/python2.7/site-packages']
I would remove the current virtualenv, recreate it without sudo and then install all requirements with the virtualenv's pip.
cd ~/Downloads
rm -rf neon/.venv/
virtualenv neon/.venv/ -p python2.7
source neon/.venv/bin/activate
pip install -r neon/requirements.txt

For what it's worth, I uninstalled pytools using pip in my main environment. Then, went into my virtual environment and used pip to install pytools. Finally, went back to my main environment and reinstalled pytools using pip install.
Now things are working. (Though oddly, pytools is no longer installed in a '.egg' directory. I think this is better, but am now even more puzzled as to why/when things get installed into .egg directories)

How to solve importing python packages " No module named "?

I am coming across a strange problem, I already installed python packages but it doesn't recognize them. please note that some packages are recognizable.
An example of importing a package
ActivePython 2.7.5.6 (ActiveState Software Inc.) based on
Python 2.7.5 (default, Sep 16 2013, 23:07:15)
[GCC 4.2.1 (Apple Inc. build 5664)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import requests
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named requests
Whenever I try to install it using Pip like
sudo pip install requests
I got this message whenever I try to install the same package.
Requirement already satisfied (use --upgrade to upgrade): requests in /usr/local/lib/python2.7/site-packages
What should I do to solve this problem ?
UPDATE_1
pip --version
pip 7.1.2 from /usr/local/lib/python2.7/site-packages

Find where pip installs your package and add the path to
export PYTHONPATH=<path>
#my path was /Library/Python/2.7/site-packages
in your .bashrc or .profile file.

psycopg2 module cannot be found by Python2.7

I installed psycopg2 via pip, but my programs are having trouble finding it.
So, I tried to install psycopg2 via pip again:
user#ubuntu:~/Desktop/progFolder$ sudo pip install psycopg2
Requirement already satisfied (use --upgrade to upgrade): psycopg2 in /usr/local/lib/python2.7/dist-packages
Cleaning up...
Then I tried to use a program that imports it:
user#ubuntu:~/Desktop/progFolder$ python myProg.py
Traceback (most recent call last):
File "myProg.py", line 6, in <module>
import psycopg2
ImportError: No module named psycopg2
And I have tried just importing directly in python:
user#ubuntu:~/Desktop/progFolder$ python
Python 2.7.5 (default, Nov 9 2014, 14:14:12)
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import psycopg2
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named psycopg2
So I printed my python path.
>>> import sys
>>> print sys.path
['', '/usr/local/lib/python27.zip', '/usr/local/lib/python2.7', '/usr/local/lib/python2.7/plat-linux2', '/usr/local/lib/python2.7/lib-tk', '/usr/local/lib/python2.7/lib-old', '/usr/local/lib/python2.7/lib-dynload', '/usr/local/lib/python2.7/site-packages']
And noticed that the path does contain the path to psycopg2.
psycopg2 in /usr/local/lib/python2.7/dist-packages
So, I have no idea on why this is happening. Any help would be appreciated.
UPDATE:
I have done
>>>help()
>>>modules
And psycopg2 was not listed among the other modules. (this does not help me but may help you help me)

Your pip looks ok (that is, it's the system/default one). Your Python executable, however, is something that didn't come by default with 14.04 LTS (e.g., on my 14.04 system, it's /usr/bin/python). Did you install that Python yourself? Then you need to install (and use) the corresponding pip as well. (Normally, Python would have come with a pip installation, but apparently in this case, it didn't.)
pip can be fairly simple installed from its installation instructions.
Though first, verify that
you did install /usr/local/bin/python yourself. That is, it didn't come with some other piece of software that you installed and that, along the way, decided to install Python there.
you want to use /usr/local/bin/python (I guess it is a more recent version of Python 2.7; the default 14.04 LTS one appears to be 2.6.7 as of 2015-08-03).

From your python path print, it looks like it doesn't have /usr/local/lib/python2.7/dist-packages included in it. You can add it in one way by:
sys.path.insert(0, "/usr/local/lib/python2.7/dist-packages")

Pandas is installed but import pandas throws error

Some information about version i am using
pip -V
Result:
pip 6.1.1 from /usr/local/lib/python3.4/site-packages (python 3.4)
Next:
python -c "import sys,pip;print(sys.version,pip,pip.__version__)"
Result:
('2.7.6 (default, Mar 22 2014, 22:59:56) \n[GCC 4.8.2]', <module 'pip' from '/usr/lib/python2.7/dist-packages/pip/__init__.pyc'>, '1.5.4')
I have installed pandas by
sudo pip install pandas
sudo easy_install pandas
I can see pandas in my system using
pip list
Result
certifi (14.5.14)
nltk (3.0.2)
numpy (1.9.2)
pandas (0.16.1)
pip (6.1.1)
pyparsing (2.0.3)
python-dateutil (2.4.2)
pytz (2015.2)
setuptools (2.1)
six (1.9.0)
tornado (4.1)
But when I import it, it throws error
python
>>> import pandas
Error
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named pandas
Pandas is installed but why it is not working and how to make pandas work with my python interpreter

I don't see why you expect your Python 2 and Python 3 installations to be in sync. The output from pip -V clearly displays that it is running on Python 3.4, and the output of python -c "import sys,pip;print(sys.version,pip,pip.__version__)" clearly indicates it is using Python 2.7 (look at the version number in the library path).
By contrast, here's the same thing using a current virtual environment on my development system.
(dojo) airhead-2:tools sholden$ pip -V
pip 1.5.6 from /Users/sholden/dojo/lib/python3.4/site-packages (python 3.4)
(dojo) airhead-2:tools sholden$ python -c "import sys,pip;print(sys.version,pip,pip.__version__)"
3.4.2 (default, Oct 19 2014, 17:50:17)
[GCC 4.2.1 Compatible Apple LLVM 5.1 (clang-503.0.40)] <module 'pip' from '/Users/sholden/dojo/lib/python3.4/site-packages/pip/__init__.py'> 1.5.6
You will see that the which command shows that pip and python both come from the same binary directory, and that the versions are matched.
(dojo) airhead-2:tools sholden$ which pip
/Users/sholden/dojo/bin/pip
(dojo) airhead-2:tools sholden$ which python
/Users/sholden/dojo/bin/python
Virtual environments were invented to keep the dependencies of one project from leaking into other projects. I suggest you think about starting to use them.

Can't install module with python-pip properly

I would like to install a module but pip is not installing it in the right directory which I assume should be /usr/local/lib/python2.7/site-packages/. After all, I just installed Python 2.7.2 today. Originally I had 2.6.5 and had installed modules successfully there. So I think something is wrong with my Python path.
How to have all my module installations go to the proper python2.7 directory?
s3z#s3z-laptop:~$ pip install requests
Requirement already satisfied: requests in /usr/local/lib/python2.6/dist-packages/requests-0.6.1-py2.6.egg
Installing collected packages: requests
Successfully installed requests
s3z#s3z-laptop:~$ python
Python 2.7.2 (default, Oct 1 2011, 14:26:08)
[GCC 4.4.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import requests
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named requests
>>> ^Z
[3]+ Stopped python
Also here is what my Python directories look like now http://pastie.org/2623543

After you installed Python 2.7, did you install the Python 2.7 version of easy_install and PIP? The existing installations are configured to use Python 2.6 by default which may be causing your issue.

You are probably using pip linked to python2.6, instead of 2.7. If you have installed pip properly with python2.7, you can do:
pip-2.7 install requests
If not, try installing this way:
curl -O http://python-distribute.org/distribute_setup.py
[sudo] python2.7 distribute_setup.py
curl -O https://raw.github.com/pypa/pip/master/contrib/get-pip.py
[sudo] python2.7 get-pip.py