Develop Reference

what is the best way to solve some Python packages that could not installed by conda

Recently I am facing a embrassing problem is that some Python packages must install using pip. I have read the pip and conda difference:
What is the difference between pip and conda?
I think many people facing the same problem that when you choose conda, then some package could not using conda install, when you use pip install the pacakge, it could not imported into the local project because the project only could import pakcage installed by conda install command. For example I want to use eyeD3 to read the mp3 meta data, but the package could not installed by conda, I installed the package using pip but could not import in the PyCharm IDE. What is the best way to solve this conflict? Am I missing something? Or I have to switch to pip? is it possible to make conda could install package from pip? Any suggestion is welcome, I want to fetch a guidline. I have tried install package like this in the macOS Big Sur:
/usr/local/anaconda3/envs/pydolphin ⌚ 22:20:21
$ ./bin/python -m pip install eyed3 ‹ruby-2.7.2›
Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple
Requirement already satisfied: eyed3 in ./lib/python3.9/site-packages (0.9.6)
Requirement already satisfied: filetype<2.0.0,>=1.0.7 in ./lib/python3.9/site-packages (from eyed3) (1.0.8)
Requirement already satisfied: coverage[toml]<6.0.0,>=5.3.1 in ./lib/python3.9/site-packages (from eyed3) (5.5)
Requirement already satisfied: deprecation<3.0.0,>=2.1.0 in ./lib/python3.9/site-packages (from eyed3) (2.1.0)
Requirement already satisfied: toml in ./lib/python3.9/site-packages (from coverage[toml]<6.0.0,>=5.3.1->eyed3) (0.10.2)
Requirement already satisfied: packaging in ./lib/python3.9/site-packages (from deprecation<3.0.0,>=2.1.0->eyed3) (21.3)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in ./lib/python3.9/site-packages (from packaging->deprecation<3.0.0,>=2.1.0->eyed3) (3.0.6)
(base)
still could not import this pakacage in the PyCharm IDE. I could found the package from the env:
/usr/local/anaconda3/envs/pydolphin/lib/python3.9/site-packages/eyed3

pipenv is installed, but $ pipenv --version shows as command not found

Newbie here not understanding how pipenv can simultaneously show as installed and not installed at the same time. Am using Windows 10 (2004) in the VS Code terminal:
$ pip install pipenv returns this:
Requirement already satisfied: pipenv in c:\users\billa\appdata\roaming\python\python37\site-packages (2018.11.26)
Requirement already satisfied: virtualenv-clone>=0.2.5 in c:\users\billa\appdata\roaming\python\python37\site-packages (from pipenv) (0.5.3)
Requirement already satisfied: certifi in c:\users\billa\appdata\local\programs\python\python37\lib\site-packages (from pipenv) (2019.6.16)
Requirement already satisfied: virtualenv in c:\users\billa\appdata\local\programs\python\python37\lib\site-packages (from pipenv) (16.7.7)
Requirement already satisfied: setuptools>=36.2.1 in c:\users\billa\appdata\local\programs\python\python37\lib\site-packages (from pipenv) (44.0.0)
Requirement already satisfied: pip>=9.0.1 in c:\users\billa\appdata\local\programs\python\python37\lib\site-packages (from pipenv) (20.1.1)
But:
$ pipenv --version
Returns this:
$ pipenv: command not found
Perplexingly, if I use Ubuntu in Windows terminal, it reports back properly:
$ pipenv, version 2020.6.2
Thanks in advance.

How libraries work? Trying to install mouseinfo

I'm trying to create a script for AutoKey (Linux) that uses the 'mouseinfo' module. I did:
pip3 install mouseinfo
I ran this line while I use in
~/.config/autokey/data/My_Scripts
but when I try to
import mouseinfo
in my python script, it seems it can't find it.
I don't really know how the python libraries are organised and how to properly use them.
python3 -m site
sys.path = [
'/home/myusr/.local/lib/python3.6/site-packages',
'/usr/lib/python36.zip',
'/usr/lib/python3.6',
'/usr/lib/python3.6/lib-dynload',
'/usr/local/lib/python3.6/dist-packages',
'/usr/lib/python3/dist-packages',
]
USER_BASE: '/home/myusr/.local' (exists)
USER_SITE: '/home/myusr/.local/lib/python3.6/site-packages' (exists)
ENABLE_USER_SITE: True
and after
cd /home/myusr/.local/lib/python3.6/site-packages
ls
I get a list of packages and I can see mouseinfo there. What's next?
which python3
/usr/bin/python3
Another try:
sudo apt-get install python3-venv
python3 -m venv env
source env/bin/activate
Successfully installed mouseinfo-0.1.3
pip3 install mouseinfo
Screenshot
and as you can see, the module is installed (I think).
Still, nothing.
I'm on Reolith Linux (Ubuntu latest version), using ZSh and Vim (and Python 3, of course), if it matters.
What's next?
Edit: I tried to install Xlib as well, same story, can't import it. I am obviously doing something wrong due to my Python ignorance, but can't figure out what.
Edit 2: installed virtualenv plugin for Vim, but :VirtualEnvList return 'Not an editor command'.
Edit 3: activated venv and also:
source env/bin/activate
~/.config/autokey/data/My_Scripts
pip install mouseinfo
Requirement already satisfied: mouseinfo in ./env/lib/python3.6/site-packages (0.1.3)
Requirement already satisfied: python3-Xlib; platform_system == "Linux" and python_version >= "3.0" in ./env/lib/python3.6/site-packages (from mouseinfo) (0.15)
Requirement already satisfied: Pillow>=4.0.0; python_version == "3.6" in ./env/lib/python3.6/site-packages (from mouseinfo) (7.1.1)
Requirement already satisfied: pyperclip in ./env/lib/python3.6/site-packages (from mouseinfo) (1.8.0)
pip3 install mouseinfo
Requirement already satisfied: mouseinfo in ./env/lib/python3.6/site-packages (0.1.3)
Requirement already satisfied: pyperclip in ./env/lib/python3.6/site-packages (from mouseinfo) (1.8.0)
Requirement already satisfied: Pillow>=4.0.0; python_version == "3.6" in ./env/lib/python3.6/site-packages (from mouseinfo) (7.1.1)
Requirement already satisfied: python3-Xlib; platform_system == "Linux" and python_version >= "3.0" in ./env/lib/python3.6/site-packages (from mouseinfo) (0.15)
It seems that I can import the module after running python3 in the terminal. Problem is with Vim probably. Any ideas?

As your default pip3 and python3 are in different locations you can try installing the package by first invoking your desired python installation and calling out to pip
python3 -m pip install mouseinfo

I decided to give up on Vim and use Visual Studio Code, and to use pyautogui's position instead. This way it works.

ImportError: No module named matplotlib even using pip install matplotlib

I am importing matplotlib from Jupyter Notebook but get the following error.
ImportError: No module named matplotlib
Settings:
Python 2
MacBook
conda: 4.3.30
After run
conda list
it got:
ipykernel 4.6.1
ipython 5.3.0
matplotlib 2.0.2 np112py36_0
matplotlib 2.1.1
Some commends I tried but failed
1.
sudo apt-get install python-matplotlib
and shows
sudo: apt-get: command not found
2.
sudo apt-get install python3-matplotlib
and shows
sudo: apt-get: command not found
commends 1 and 2 are from this site
ImportError: No module named matplotlib with matplotlib installed
3.
pip install matplotlib
and shows
Requirement already satisfied: matplotlib in ./anaconda/lib/python3.6/site-packages
Requirement already satisfied: pytz in ./anaconda/lib/python3.6/site-packages (from matplotlib)
Requirement already satisfied: cycler>=0.10 in ./anaconda/lib/python3.6/site-packages (from matplotlib)
Requirement already satisfied: numpy>=1.7.1 in ./anaconda/lib/python3.6/site-packages (from matplotlib)
Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in ./anaconda/lib/python3.6/site-packages (from matplotlib)
Requirement already satisfied: six>=1.10 in ./anaconda/lib/python3.6/site-packages (from matplotlib)
Requirement already satisfied: python-dateutil>=2.0 in ./anaconda/lib/python3.6/site-packages (from matplotlib)
commend 3 from this website
ImportError: No module named matplotlib.pyplot
Update:
run this
pip2 install matplotlib
it was successfully installed like this
Collecting matplotlib
Downloading matplotlib-2.1.1-cp27-cp27m...
...
Successfully built subprocess32
Installing collected packages: subprocess32, six, python-dateutil, backports.functools-lru-cache, pyparsing, cycler, numpy, pytz, matplotlib
Successfully installed backports.functools-lru-cache-1.4 cycler-0.10.0 matplotlib-2.1.1 numpy-1.13.3 pyparsing-2.2.0 python-dateutil-2.6.1 pytz-2017.3 six-1.11.0 subprocess32-3.2.7
then restart Anaconda and Jupyter, and it was not imported.

By the looks of it you already have matplot lib installed for python3.6, just run with python3.6 instead of python, which ends up running python2.
pip will also normally install packages for python2 so if you want to install them for python3 use pip3 install <package>
since apt-get can not be found you are probably running CentOS which uses yum instead as the default package manager. To install you'd normally use the following command:
yum install <packagename>

Unable to update pip3 even though I try to update it

In summary, it seems I get into a loop where pip thinks I am using an outdated version of pip but when I try to upgrade it does NOT allow me. So I attempt an installation (from this question Jupyter Notebook can't find modules for python 3.6):
(automl) brandomiranda~ ❯ pip3 install ipykernel --upgrade
python3 -m ipykernel install --user
Requirement already up-to-date: ipykernel in /usr/local/lib/python3.6/site-packages (5.1.3)
Requirement not upgraded as not directly required: jupyter-client in /usr/local/lib/python3.6/site-packages (from ipykernel) (5.3.4)
Requirement not upgraded as not directly required: traitlets>=4.1.0 in /usr/local/lib/python3.6/site-packages (from ipykernel) (4.3.3)
Requirement not upgraded as not directly required: appnope; platform_system == "Darwin" in /usr/local/lib/python3.6/site-packages (from ipykernel) (0.1.0)
Requirement not upgraded as not directly required: tornado>=4.2 in /usr/local/lib/python3.6/site-packages (from ipykernel) (6.0.3)
Requirement not upgraded as not directly required: ipython>=5.0.0 in /usr/local/lib/python3.6/site-packages (from ipykernel) (7.11.1)
Requirement not upgraded as not directly required: jupyter-core>=4.6.0 in /usr/local/lib/python3.6/site-packages (from jupyter-client->ipykernel) (4.6.1)
Requirement not upgraded as not directly required: pyzmq>=13 in /usr/local/lib/python3.6/site-packages (from jupyter-client->ipykernel) (18.1.1)
Requirement not upgraded as not directly required: python-dateutil>=2.1 in /usr/local/lib/python3.6/site-packages (from jupyter-client->ipykernel) (2.8.1)
Requirement not upgraded as not directly required: six in /usr/local/lib/python3.6/site-packages (from traitlets>=4.1.0->ipykernel) (1.11.0)
Requirement not upgraded as not directly required: ipython-genutils in /usr/local/lib/python3.6/site-packages (from traitlets>=4.1.0->ipykernel) (0.2.0)
Requirement not upgraded as not directly required: decorator in /usr/local/lib/python3.6/site-packages (from traitlets>=4.1.0->ipykernel) (4.4.1)
Requirement not upgraded as not directly required: backcall in /usr/local/lib/python3.6/site-packages (from ipython>=5.0.0->ipykernel) (0.1.0)
Requirement not upgraded as not directly required: pickleshare in /usr/local/lib/python3.6/site-packages (from ipython>=5.0.0->ipykernel) (0.7.5)
Requirement not upgraded as not directly required: pexpect; sys_platform != "win32" in /usr/local/lib/python3.6/site-packages (from ipython>=5.0.0->ipykernel) (4.7.0)
Requirement not upgraded as not directly required: prompt-toolkit!=3.0.0,!=3.0.1,<3.1.0,>=2.0.0 in /usr/local/lib/python3.6/site-packages (from ipython>=5.0.0->ipykernel) (3.0.2)
Requirement not upgraded as not directly required: jedi>=0.10 in /usr/local/lib/python3.6/site-packages (from ipython>=5.0.0->ipykernel) (0.15.2)
Requirement not upgraded as not directly required: pygments in /usr/local/lib/python3.6/site-packages (from ipython>=5.0.0->ipykernel) (2.5.2)
Requirement not upgraded as not directly required: setuptools>=18.5 in /usr/local/lib/python3.6/site-packages (from ipython>=5.0.0->ipykernel) (39.0.1)
Requirement not upgraded as not directly required: ptyprocess>=0.5 in /usr/local/lib/python3.6/site-packages (from pexpect; sys_platform != "win32"->ipython>=5.0.0->ipykernel) (0.6.0)
Requirement not upgraded as not directly required: wcwidth in /usr/local/lib/python3.6/site-packages (from prompt-toolkit!=3.0.0,!=3.0.1,<3.1.0,>=2.0.0->ipython>=5.0.0->ipykernel) (0.1.8)
Requirement not upgraded as not directly required: parso>=0.5.2 in /usr/local/lib/python3.6/site-packages (from jedi>=0.10->ipython>=5.0.0->ipykernel) (0.5.2)
You are using pip version 10.0.1, however version 19.3.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
Installed kernelspec python3 in /Users/brandomiranda/Library/Jupyter/kernels/python3
ok fine, I will upgrade but then it thinks its already upgraded:
(automl) brandomiranda~ ❯ pip install --upgrade pip
Requirement already up-to-date: pip in ./miniconda3/envs/automl/lib/python3.7/site-packages (19.3.1)
but when I try the installation again it fails with the same message. It seems like a chicken and egg problem. Anyone know what's going on?
Outdated:
I recently did a pip3 list to check what modules/projects I had installed in my virtual environment and I got a suggestion to upgrade pip. I tried but I can't seem to make it update/upgrade. I did:
$ pip3 list
namespaces (4.2.0)
pip (8.0.2)
scikit-learn (0.18.1)
setuptools (19.4)
six (1.10.0)
wheel (0.26.0)
You are using pip version 8.0.2, however version 9.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
so I tried doing that and it said it was working:
$ pip install --upgrade pip
Requirement already up-to-date: pip in /Users/user/home_simulation_research/hbf_tensorflow_code/venv/lib/python2.7/site-packages
since it said its alread there I did pip3 list again and it kept saying I didn't actually have it upgraded. Since updating with the standard pip command didn't work I instead tried using pip3 to do the update but it still didn't work:
pip3 install --upgrade pip3
Collecting pip3
Could not find a version that satisfies the requirement pip3 (from versions: )
No matching distribution found for pip3
You are using pip version 8.0.2, however version 9.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
Why doesn't it work?
Just when I thought it didn't work I tried using pip3 to upgrade pip and pip3 to upgrade pip3. For some reason the later doesn't seem to work but the first one does. i.e. doing:
$ pip3 install --upgrade pip
worked (or at least pip3 list) stopped requesting me upgrade. Why is that? Is it just me or is this really strange behaviour?

when you run pip3 list, you are listing packages installed for python3, and when you run pip install --upgrade pip, you are checking updates for python2 packages. when you run pip3 install --upgrade pip3, this would give an error because there is not package named pip3, pip3 is a command line tool provided in the pip package. When you finally run pip3 install --upgrade pip, you are doing the right thing(installing updates for python3, package name is pip).

What worked for me was to uninstall and install pip.
To uninstall pip do:
pip uninstall pip
although I went to the /usr/local/bin and deleted everything with pip name with:
rm -rf pip*
there were some pip3 things that were still there oddly enough.
Then I install pip with the (deprecated) sudo easy install:
sudo easy_install pip
although that command is deprecated, so something else has to be done in mac but for now that works (and made my jupyter notebook work again thus solving this question Jupyter Notebook can't find modules for python 3.6).

If you notice the difference in site-packages path you will get it.
When you are using pip3 /usr/local/lib/python3.6/site-packages path is shown which is python3.6.
Ref:
(automl) brandomiranda~ ❯ pip3 install ipykernel --upgrade
python3 -m ipykernel install --user
Requirement already up-to-date: ipykernel in /usr/local/lib/python3.6/site-packages (5.1.3)
And when you are using just pip you see it is python3.7 and totally different site-packages path ./miniconda3/envs/automl/lib/python3.7/site-packages
Ref:
(automl) brandomiranda~ ❯ pip install --upgrade pip
Requirement already up-to-date: pip in ./miniconda3/envs/automl/lib/python3.7/site-packages (19.3.1)
So both are different : pip3 points to direct system path(/usr/local/lib docs) which is Python3.6 and pip points to a virtual environment's path(./miniconda3/envs/automl/lib/) where python3.7 is installed.
You need to decide which one to upgrade and use. Having virtual environment is best practice.

The issue seems to be that new virtual environments are using an old version of pip. Note that pip is installed from a source tarfile (or wheel) included with virtualenv, in the site-packages/virtualenv_support directory.
$ ls -l /path/to/site-packages/virtualenv_support
pip-9.1-py2.py3-none-any.whl
A quick way to workaround the problem is to make sure you upgrade pip whenever you create a new virtualenv, like so:
$ virtualenv venv
$ venv/bin/pip install -U pip
Alternatively, make sure you have the latest version of virtualenv. According to their release notes, virtualenv==16 is using pip==10.
$ pip install -U virtualenv
Finally, since virtualenv looks for pip*.whl in virtualenv_support, this will also work:
$ mv /path/to/site-packages/virtualenv_support/pip*.whl{,bak}
$ pip wheel -w /path/to/site-packages/virtualenv_support/ 'pip==18'
All new virtualenvs will use the version of pip that you installed into virtualenv_support. However, this feels hacky.
(Attempted with virtualenv==16. This results in all new virtualenvs with pip==18.)
Stackoverflow Post - https://stackoverflow.com/questions/51644402/i-keep-getting-a-message-to-upgrade-pip

instead of pip install, I tend to run python3 -m pip install upgrade --user, to specify which python is intended

To get me out of a similar loop of hell, I simply re-installed pip with the following command :
curl https://bootstrap.pypa.io/get-pip.py | python3.7
Just specify the right python version you want pip to be installed with.
Url is from https://pip.pypa.io/en/stable/installing/