Develop Reference

Can't import module installed with pip (anaconda python)

I'm trying to use the ReadIM package with an anaconda environment (anaconda 4.7.12, python 3.8, pip 19.3.1), thus requiring me to use pip install readIM which succeeds, with the package found in conda list and pip list.
I have read a lot of posts talking about using the correct python and pip paths, my python where shows only my anaconda path, where I ran conda install pip beforehand, and then pip -V shows my pip is in my conda path.
I have tried conda update --all and pip uninstall readIM. I also read that it could be an issue with Jupyter notebook, hence in the conda command prompt I tried python then import readIM, which still gave me the same import error.
As advised on the source page, I tried installing by running python setup.py build install/python setup.py install in the source directory, however this gave me an error
cygwin TypeError: '>=' not supported between instances of 'NoneType' and 'str'
I am at a loss, any ideas?
Thanks,
Mustafa.
UPDATE:
Managed to compile it from source successfully, but still get the same error:
(base) C:>python -m pip install ReadIM-0.8.2.tar.gz
Processing c:\readim-0.8.2.tar.gz
Building wheels for collected packages: ReadIM
Building wheel for ReadIM (setup.py) ... done
Created wheel for ReadIM: filename=ReadIM-0.8.2-cp37-cp37m-win_amd64.whl size=219546 sha256=90f8960a6f1f80ae62dc18eab4bcd31fb2a6dfd7da364a5c15fc37e6e2ce0360
Stored in directory: C:\Users\mi4517\AppData\Local\pip\Cache\wheels\d6\a9\11\936e986255027bb654601b322a3431f9bcc3fde72ebb406835
Successfully built ReadIM
Installing collected packages: ReadIM
Successfully installed ReadIM-0.8.2
(base) C:>python
Python 3.7.5 (default, Oct 31 2019, 15:18:51) [MSC v.1916 64 bit (AMD64)] :: Anaconda, Inc. on win32
Type "help", "copyright", "credits" or "license" for more information.
import readIM
Traceback (most recent call last):
File "", line 1, in
ModuleNotFoundError: No module named 'readIM'>
Following their instructions (python setup.py build install then python setup.py test) yields a successful build but a failed test:
C:\ReadIM-0.8.2>python setup.py test
Traceback (most recent call last):
File "setup.py", line 41, in
assert os.path.isdir(testFileDir)
AssertionError
SOLUTION
It was installing correctly, just that I was importing it incorrectly in Python. I was doing import readIM or import readim when it should be import ReadIM I did not know it was case sensitive!
Thank you for your help, apologies for the trivial mistake on my part.

I had faced a similar issue. With your conda environment activated, type
which pip
Check if it shows the path to pip in your environment. In my case, it was not. It was showing path to some other pip. So pip install was installing in that environment. To fix use the full path of pip, like below
~/anaconda3/envs/my_env/bin/pip install ...

To avoid this issue follow these steps
open Anaconda prompt
type this command conda install -c conda-forge imread
after installation open jupiter notebook.
type import imread then run.

It was installing correctly, just that I was importing it incorrectly in Python. I was doing import readIM or import readim when it should be import ReadIM I did not know it was case sensitive!
Thank you for your help, apologies for the trivial mistake on my part.

Python Modules Not In Correct Location

We have CentOS 7 servers in a Hadoop cluster. Python and pip should have been installed on all of the server the same way because it was done with Ansible. But for some reason, there are some servers where Python cannot use the modules that were installed. For instance pandas was install, but when in python3.6, I get a 'No module named pandas' error.
$ python3.6
Python 3.6.2 (default, Aug 2 2017, 14:51:00)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-11)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pandas
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'pandas'
>>>
If I try installing pandas again, I get messages that the reuirements are already satisfied.
$ sudo pip3.6 install pandas
Requirement already satisfied: pandas in /usr/lib64/python3.6/site-packages
Requirement already satisfied: python-dateutil>=2 in /usr/lib/python3.6/site-packages (from pandas)
Requirement already satisfied: pytz>=2011k in /usr/lib/python3.6/site-packages (from pandas)
Requirement already satisfied: numpy>=1.9.0 in /usr/lib64/python3.6/site-packages (from pandas)
Requirement already satisfied: six>=1.5 in /usr/lib/python3.6/site-packages (from python-dateutil>=2->pandas)
It appears that the python 3.6 sys.path is using a different location for site-packages.
$ python3.6 -m site
sys.path = [
'/home/avalenti',
'/usr/local/lib/python36.zip',
'/usr/local/lib/python3.6',
'/usr/local/lib/python3.6/lib-dynload',
'/usr/local/lib/python3.6/site-packages',
]
It may all stem from pip being installed in an unusual location.
$ pip3.6 --version
pip 9.0.1 from /usr/lib/python3.6/site-packages (python 3.6)
Pip and the modules seem to be located in the same place, correctly on servers that DO work as follows:
$ pip3.6 --version
pip 9.0.1 from /usr/local/lib/python3.6/site-packages (python 3.6)
$ python3.6 -m site
sys.path = [
'/home/avalenti',
'/usr/local/lib/python36.zip',
'/usr/local/lib/python3.6',
'/usr/local/lib/python3.6/lib-dynload',
'/usr/local/lib/python3.6/site-packages',
]
$ python3.6
Python 3.6.2 (default, Aug 2 2017, 14:17:20)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-11)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pandas
>>> exit()
$ pip3.6 show pandas
Name: pandas
Version: 0.22.0
Summary: Powerful data structures for data analysis, time series,and statistics
Home-page: http://pandas.pydata.org
Author: The PyData Development Team
Author-email: pydata#googlegroups.com
License: BSD
Location: /usr/local/lib/python3.6/site-packages
Requires: numpy, pytz, python-dateutil
So my question is, how do I get this fixed and how do I keep this from happening in the first place? I don't seem to find a way to configure Pip and I don't find any way to specify the installation location for modules.
I tried to be thorough with the information provided, but I'm sure I missed something.
Thanks,
Anthony
Well, the Pip documentation for the config pointed me to the --target option for pip install. That allowed me to specify the location where I thought the packages should go - /usr/local/lib/python3.6/site-packages (as noted in the sys.path). That allowed the import to find the pandas package, but fail on the import of some basic system modules, builtin and binascii. It seems like python itself is kinda hosed.
# pip3.6 install --target /usr/local/lib/python3.6/site-packages pandas
Collecting pandas
Using cached pandas-0.22.0-cp36-cp36m-manylinux1_x86_64.whl
Collecting numpy>=1.9.0 (from pandas)
Downloading numpy-1.14.2-cp36-cp36m-manylinux1_x86_64.whl (12.2MB)
100% |████████████████████████████████| 12.2MB 92kB/s
Collecting python-dateutil>=2 (from pandas)
Downloading python_dateutil-2.7.2-py2.py3-none-any.whl (212kB)
100% |████████████████████████████████| 215kB 2.4MB/s
Collecting pytz>=2011k (from pandas)
Using cached pytz-2018.3-py2.py3-none-any.whl
Collecting six>=1.5 (from python-dateutil>=2->pandas)
Using cached six-1.11.0-py2.py3-none-any.whl
Installing collected packages: numpy, six, python-dateutil, pytz, pandas
Successfully installed numpy-1.14.2 pandas-0.22.0 python-dateutil-2.7.2 pytz-2018.3 six-1.11.0
You are using pip version 9.0.1, however version 9.0.3 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
# python3.6
Python 3.6.2 (default, Aug 2 2017, 14:51:00)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-11)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pandas
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/pandas/compat/__init__.py", line 48, in <module>
import __builtin__ as builtins
ModuleNotFoundError: No module named '__builtin__'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python3.6/site-packages/pandas/__init__.py", line 23, in <module>
from pandas.compat.numpy import *
File "/usr/local/lib/python3.6/site-packages/pandas/compat/__init__.py", line 61, in <module>
import http.client as httplib
File "/usr/local/lib/python3.6/http/client.py", line 71, in <module>
import email.parser
File "/usr/local/lib/python3.6/email/parser.py", line 12, in <module>
from email.feedparser import FeedParser, BytesFeedParser
File "/usr/local/lib/python3.6/email/feedparser.py", line 27, in <module>
from email._policybase import compat32
File "/usr/local/lib/python3.6/email/_policybase.py", line 7, in <module>
from email import header
File "/usr/local/lib/python3.6/email/header.py", line 14, in <module>
import binascii
ModuleNotFoundError: No module named 'binascii'
>>> exit()
Ugh...

You need to be very careful using pip when having multiple Python distributions. Instead of using pip3.6', try usingpython3.6 -m pip` to install pandas:
python3.6 -m pip install pandas
(prepend sudo if you really need to). In this way you are invoking pip through Python itself, and so you are guaranteed to get the pip that belongs with the Python you want.

You can try to set the location for pip in $HOME/.pip/pip.conf
As per PIP Docs:
Configuration Config file
pip allows you to set all command line option defaults in a standard
ini style config file.
The names and locations of the configuration files vary slightly
across platforms. You may have per-user, per-virtualenv or site-wide
(shared amongst all users) configuration:
Per-user:
On Unix the default configuration file is: $HOME/.config/pip/pip.conf which respects the XDG_CONFIG_HOME
environment variable.
On macOS the configuration file is $HOME/Library/Application Support/pip/pip.conf.
On Windows the configuration file is %APPDATA%\pip\pip.ini.
There are also a legacy per-user configuration file which is also
respected, these are located at:
On Unix and macOS the configuration file is: $HOME/.pip/pip.conf
On Windows the configuration file is: %HOME%\pip\pip.ini
You can set a custom path location for this config file using the
environment variable PIP_CONFIG_FILE.
Inside a virtualenv:
On Unix and macOS the file is $VIRTUAL_ENV/pip.conf
On Windows the file is: %VIRTUAL_ENV%\pip.ini
Site-wide:
On Unix the file may be located in /etc/pip.conf. Alternatively it may be in a "pip" subdirectory of any of the paths set in the
environment variable XDG_CONFIG_DIRS (if it exists), for example
/etc/xdg/pip/pip.conf.
On macOS the file is: /Library/Application Support/pip/pip.conf
On Windows XP the file is: C:\Documents and Settings\All Users\Application Data\pip\pip.ini
On Windows 7 and later the file is hidden, but writeable at C:\ProgramData\pip\pip.ini
Site-wide configuration is not supported on Windows Vista
If multiple configuration files are found by pip then they are
combined in the following order:
Firstly the site-wide file is read, then
The per-user file is read, and finally
The virtualenv-specific file is read.
Each file read overrides any values read from previous files, so if
the global timeout is specified in both the site-wide file and the
per-user file then the latter value is the one that will be used.

Short answer, but I would recommend using Python Virtual Environments. Makes managing Python versions and pip packages very easy.
https://docs.python.org/3/tutorial/venv.html
This way you can have many different virtualized Python environments and activate them for whatever script, Ansible playbook, etc you are running. Another great feature is you can create a requirements file to share with others, so if they want to run your code they can simply install from the requirements file which guarantees they have the right version of Python and associated packages installed.

Which pip is with which python?

This is driving me crazy. I have tried to remove all the packages of python installed on mac os x el capitan and re-installed brew install python and pip. Here I have :
which pip
/usr/local/Cellar/python/2.7.11/Frameworks/Python.framework/Versions/2.7/bin/pip
and
which python
/usr/local/Cellar/python/2.7.11/Frameworks/Python.framework/Versions/2.7/bin/python
so using pip list
pip list
cycler (0.10.0)
matplotlib (1.5.1)
numpy (1.11.0)
pip (8.1.2)
python-dateutil (2.5.3)
pytz (2016.4)
setuptools (19.4)
six (1.10.0)
wheel (0.26.0)
However, when I run python, there is no module called numpy and matplotlib:
python
Python 2.7.11 (default, Jan 22 2016, 08:29:18)
[GCC 4.2.1 Compatible Apple LLVM 7.0.2 (clang-700.1.81)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named numpy
>>> import matplotlib
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python2.7/site-packages/matplotlib/__init__.py", line 122, in <module>
from matplotlib.cbook import is_string_like, mplDeprecation, dedent, get_label
File "/usr/local/lib/python2.7/site-packages/matplotlib/cbook.py", line 33, in <module>
import numpy as np
ImportError: No module named numpy
and this is my sys.path:
sys.path
['', '/usr/local/Cellar/python/2.7.11/Frameworks/Python.framework/Versions/2.7/lib/python27.zip', '/usr/local/Cellar/python/2.7.11/Frameworks/Python.framework/Versions/2.7/lib/python2.7', '/usr/local/Cellar/python/2.7.11/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-darwin', '/usr/local/Cellar/python/2.7.11/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac', '/usr/local/Cellar/python/2.7.11/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac/lib-scriptpackages', '/usr/local/Cellar/python/2.7.11/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-tk', '/usr/local/Cellar/python/2.7.11/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-old', '/usr/local/Cellar/python/2.7.11/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-dynload', '/usr/local/lib/python2.7/site-packages', '/usr/local/lib/python2.7/site-packages/gtk-2.0']

You can run pip with a specific version of Python by running it as a module. Command line arguments work just as if running directly from the command line. For example, try:
python -m pip list
If that still lists numpy it probably means there is something wrong with the numpy installation — i.e. the .egg file is there, but the module folder is not. To try and fix this you can use --force-reinstall with pip, e.g.
python -m pip install numpy --force-reinstall --upgrade
If that still doesn't work, you can resort to going to the folder reported by sys.path and deleting anything numpy related manually.
Since your pip setup seems messed up you might want to try reinstalling pip too!

The most accurate method is to call pip from the specific python executable. For example:
/usr/bin/python -m pip list -v
/usr/local/bin/python3 -m pip list -v

psycopg2 module cannot be found by Python2.7

I installed psycopg2 via pip, but my programs are having trouble finding it.
So, I tried to install psycopg2 via pip again:
user#ubuntu:~/Desktop/progFolder$ sudo pip install psycopg2
Requirement already satisfied (use --upgrade to upgrade): psycopg2 in /usr/local/lib/python2.7/dist-packages
Cleaning up...
Then I tried to use a program that imports it:
user#ubuntu:~/Desktop/progFolder$ python myProg.py
Traceback (most recent call last):
File "myProg.py", line 6, in <module>
import psycopg2
ImportError: No module named psycopg2
And I have tried just importing directly in python:
user#ubuntu:~/Desktop/progFolder$ python
Python 2.7.5 (default, Nov 9 2014, 14:14:12)
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import psycopg2
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named psycopg2
So I printed my python path.
>>> import sys
>>> print sys.path
['', '/usr/local/lib/python27.zip', '/usr/local/lib/python2.7', '/usr/local/lib/python2.7/plat-linux2', '/usr/local/lib/python2.7/lib-tk', '/usr/local/lib/python2.7/lib-old', '/usr/local/lib/python2.7/lib-dynload', '/usr/local/lib/python2.7/site-packages']
And noticed that the path does contain the path to psycopg2.
psycopg2 in /usr/local/lib/python2.7/dist-packages
So, I have no idea on why this is happening. Any help would be appreciated.
UPDATE:
I have done
>>>help()
>>>modules
And psycopg2 was not listed among the other modules. (this does not help me but may help you help me)

Your pip looks ok (that is, it's the system/default one). Your Python executable, however, is something that didn't come by default with 14.04 LTS (e.g., on my 14.04 system, it's /usr/bin/python). Did you install that Python yourself? Then you need to install (and use) the corresponding pip as well. (Normally, Python would have come with a pip installation, but apparently in this case, it didn't.)
pip can be fairly simple installed from its installation instructions.
Though first, verify that
you did install /usr/local/bin/python yourself. That is, it didn't come with some other piece of software that you installed and that, along the way, decided to install Python there.
you want to use /usr/local/bin/python (I guess it is a more recent version of Python 2.7; the default 14.04 LTS one appears to be 2.6.7 as of 2015-08-03).

From your python path print, it looks like it doesn't have /usr/local/lib/python2.7/dist-packages included in it. You can add it in one way by:
sys.path.insert(0, "/usr/local/lib/python2.7/dist-packages")

Failed to load straight.plugin in Python 2.7

I am trying to run a python script which is seems to be dependent on a module called straight.plugin, which it fails to recognise.
(antismash)[username#glenn antismash]$ python run_antismash.py
Traceback (most recent call last):
File "run_antismash.py", line 32, in <module>
import straight.plugin
File "/c3se/users/username/Glenn/virtualenvs/antismash/lib/python2.7/site-packages/straight/plugin/__init__.py", line 1, in <module>
from straight.plugin import loaders
File "/c3se/users/username/Glenn/virtualenvs/antismash/lib/python2.7/site-packages/straight/plugin/loaders.py", line 9, in <module>
from straight.plugin.manager import PluginManager
ImportError: No module named manager
When I try to install straight.plugin python tells me that it is already installed.
(antismash)[username#glenn antismash]$ pip instal
l straight.plugin
Requirement already satisfied (use --upgrade to upgrade): straight.plugin in /c3se/users/username/Glenn/virtualenvs/antismash/lib/python2.7/site-packages
Cleaning up...
Can somebody help me on what to do to run this python script?
I am running Python 2.7.6 on Linux distribution CentOS 6.6

Looks like the manager module isnt in the PyPI Package and is a known issue https://github.com/ironfroggy/straight.plugin/issues/17
Installing this version:
pip install straight.plugin==1.4.0-post-1
solves the problem for me:
In [1]: from straight.plugin.manager import PluginManager
In [2]: