Develop Reference

Python packages are located in differente location

I am using Ubuntu 20.04 which comes python 3.8.
I installed python3.9 and made it as default python update-alternatives
under /usr/bin/ I have python2.7/ python3/ python3.8/ and python3.9/
==> Now the problem that I am facing is when I installed sqlalchemy using pip install sqlalchemy its version was 1.3.12, which has a compatibility problem with pandas because it requires version '1.4.0'.
>>>pip install --upgrade sqlalchemy
>>> Defaulting to user installation because normal site-packages is not writeable
Requirement already satisfied: sqlalchemy in /usr/lib/python3/dist-packages (1.3.12)
Collecting sqlalchemy
Downloading SQLAlchemy-1.4.31-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.6/1.6 MB 8.3 MB/s eta 0:00:00
Collecting greenlet!=0.4.17
Downloading greenlet-1.1.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (156 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 156.6/156.6 KB 9.6 MB/s eta 0:00:00
Installing collected packages: greenlet, sqlalchemy
Successfully installed greenlet-1.1.2 sqlalchemy-1.4.31
I wrote the same command again it gave me this :
Defaulting to user installation because normal site-packages is not writeable
Requirement already satisfied: sqlalchemy in ./.local/lib/python3.8/site-packages (1.4.31)
Requirement already satisfied: greenlet!=0.4.17 in ./.local/lib/python3.8/site-packages (from sqlalchemy) (1.1.2)
I also did this to check:
>>>pip show sqlalchemy | grep Location
>>>Location: /home/Moe/.local/lib/python3.8/site-packages
but when I import sqlalchemy in a script it still uses the old version
it seems that the new version is installed in a different location
>>> print(sqlalchemy.__file__)
>>>/usr/lib/python3/dist-packages/sqlalchemy/__init__.py
Can Someone please explain this to me, and if possible explain the entire package management problem here

You are calling pip that is associated with your python 3.8, you can see that in the whl file that is downloaded:
Downloading SQLAlchemy-1.4.31-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Note the cp38
And in the output of running the install again:
Requirement already satisfied: sqlalchemy in ./.local/lib/python3.8/site-packages (1.4.31)
Note the python3.8.
Since you have mentioned that you downloaded python 3.9, you are probably running your scripts using that version of python. You will need to use the pip that is associated with that python. If you are running your script with
python <scriptfile>
Then you should pip install with
python -m pip install ...

Librosa installation pip fail

I am trying to install librosa by pip install librosa on the mac terminal, pip3 also has the same problem. It also gets stuck when I try to install scikit-learn.
Doesn't go any further than:
Collecting librosa
Using cached librosa-0.8.1-py3-none-any.whl (203 kB)
Collecting packaging>=20.0
Using cached packaging-21.0-py3-none-any.whl (40 kB)
Requirement already satisfied: numpy>=1.15.0 in /opt/homebrew/lib/python3.9/site-packages (from librosa) (1.21.0)
Requirement already satisfied: scipy>=1.0.0 in /opt/homebrew/lib/python3.9/site-packages (from librosa) (1.7.0)
Collecting numba>=0.43.0
Using cached numba-0.53.1.tar.gz (2.2 MB)
Collecting resampy>=0.2.2
Using cached resampy-0.2.2.tar.gz (323 kB)
Requirement already satisfied: soundfile>=0.10.2 in /opt/homebrew/lib/python3.9/site-packages (from librosa) (0.10.3.post1)
Collecting scikit-learn!=0.19.0,>=0.14.0
Using cached scikit-learn-0.24.2.tar.gz (7.5 MB)
Installing build dependencies ... \
I am very new at this, please help I just want to draw some spectrograms.
python --version
Python 2.7.16
python3 --version
Python 3.9.6
pip --version
pip 21.1.3 from /opt/homebrew/lib/python3.9/site-packages/pip (python 3.9)

It might be useful for you to work in a virtual environment. If you are trying to install with python 3.9 there may be a number of dependency issues, so using a lower version may also solve your issues. However, do try to with a conda environment as this will allow you to more easily keep track of install packages etc. Here are the install instructions:
After the install you can create a conda environment with a lower python version:
conda create -n "py38" python=3.8 ipython
Activate the environment:
conda activate py38
Install librosa:
conda install -c conda-forge librosa

Pip failing to install matplotlib

I had to redownload Python and PyCharm due to some issues with my old versions, and am now reinstalling some packages for my project. numpy and pandas installed fine using the PyCharm package manager interface, but when I try to install matplotlib I just get an error with message
ERROR: Command errored out with exit status 1:
When I look for more information, PyCharm suggests that I "Try to run pip install matplotlib from the system terminal. Make sure that you use the correct version of 'pip' installed for your Python interpreter located at [my path]". When I try to run the command in terminal, I get the following message:
DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. pip 21.0 will drop support for Python 2.7 in January 2021. More details about Python 2 support in pip can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support pip 21.0 will remove support for this functionality.
Defaulting to user installation because normal site-packages is not writeable
Collecting matplotlib
Using cached matplotlib-2.2.5-cp27-cp27mu-manylinux1_x86_64.whl (12.8 MB)
Collecting python-dateutil>=2.1
Using cached python_dateutil-2.8.1-py2.py3-none-any.whl (227 kB)
Collecting kiwisolver>=1.0.1
Using cached kiwisolver-1.1.0-cp27-cp27mu-manylinux1_x86_64.whl (93 kB)
Collecting numpy>=1.7.1
Using cached numpy-1.16.6-cp27-cp27mu-manylinux1_x86_64.whl (17.0 MB)
Collecting six>=1.10
Using cached six-1.15.0-py2.py3-none-any.whl (10 kB)
Collecting cycler>=0.10
Using cached cycler-0.10.0-py2.py3-none-any.whl (6.5 kB)
Collecting pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pytz
Using cached pytz-2020.4-py2.py3-none-any.whl (509 kB)
Collecting backports.functools-lru-cache
Using cached backports.functools_lru_cache-1.6.1-py2.py3-none-any.whl (5.7 kB)
Collecting subprocess32
Using cached subprocess32-3.5.4-cp27-cp27mu-manylinux2014_x86_64.whl (69 kB)
Requirement already satisfied: setuptools in /home/[myusername]/.local/lib/python2.7/site-packages (from kiwisolver>=1.0.1->matplotlib) (44.1.1)
Installing collected packages: six, python-dateutil, kiwisolver, numpy, cycler, pyparsing, pytz, backports.functools-lru-cache, subprocess32, matplotlib
Successfully installed backports.functools-lru-cache-1.6.1 cycler-0.10.0 kiwisolver-1.1.0 matplotlib-2.2.5 numpy-1.16.6 pyparsing-2.4.7 python-dateutil-2.8.1 pytz-2020.4 six-1.15.0 subprocess32-3.5.4
I'm using windows, but have Ubuntu installed, which I used to try to run that command, since when I try to do anything with pip on git bash I get bash: /c/Users/[myusername]/AppData/Local/Microsoft/WindowsApps/python: Permission denied.
Anyways, I suppose the problem is that I have Python 2.7 installed (python --version agrees), which won't cooperate with matplotlib. So, I go to /home/[myusername]/.local/lib/python2.7/site-packages using Ubuntu terminal and use rm -r python2.7 to delete folder. Since I have Python 3.9 installed at C:\Users\[myusername]\AppData\Local\Programs\Python\Python39\python.exe I assumed that this would just default to the newer version. However, when I restart everything, Python 2.7 just appears again in its old location, and that version continues to run. I believe this has something to do with how I'm using Ubuntu, since Python 2.7 exists only outside my C: drive. But, I'm not sure how to go from here to install matplotlib. Would appreciate any help!

Can't install new packages or list existing packages in virtual environment with pip

Update / Solution:
Found a fix/workaround in the answer to this question:
Pip installing packages to global site-packages when inside virtual environment.
Like the OP of that question, I'm also running Manjaro Linux with the latest updates.
When trying to install the django import_export module with pip install django-import-export into my virtual environment, it indicates that it was successfully installed, however, when I try to use the import_export module in my project, I receive an error: ModuleNotFoundError: No module named 'import_export'. I'm also unable to list any previously installed apps with pip list or pip freeze, and trying to upgrade the pip version using pip install --upgrade pip also says the new version is successfully installed, but isn't installed within my virtual environment.
I've made sure to activate my virtual environment with source /home/travis/Documents/Python/Django/Projects/issuetracker/.env/bin/activate.
Output from pip install django-import-export:
(.env) [travis#spooky issuetracker]$ pip install django-import-export
Collecting django-import-export
Using cached https://files.pythonhosted.org/packages/62/7a/ddd9aef718243504e7715bda9bb5a100cfc353be37dc819d9914a7073cba/django_import_export-1.2.0-py2.py3-none-any.whl
Collecting tablib (from django-import-export)
Using cached https://files.pythonhosted.org/packages/7b/c7/cb74031b330cd94f3580926dc707d148b4ba9138449fc9f433cb79e640d8/tablib-0.13.0-py3-none-any.whl
Collecting diff-match-patch (from django-import-export)
Using cached https://files.pythonhosted.org/packages/f0/2a/5ba07def0e9107d935aba62cf632afbd0f7c723a98af47ccbcab753d2452/diff-match-patch-20181111.tar.gz
Requirement already satisfied: django>=1.8 in ./.env/lib/python3.7/site-packages (from django-import-export) (2.2.4)
Collecting openpyxl>=2.4.0 (from tablib->django-import-export)
Using cached https://files.pythonhosted.org/packages/f5/39/942a406621c1ff0de38d7e4782991b1bac046415bf54a66655c959ee66e8/openpyxl-2.6.3.tar.gz
Collecting xlwt (from tablib->django-import-export)
Using cached https://files.pythonhosted.org/packages/44/48/def306413b25c3d01753603b1a222a011b8621aed27cd7f89cbc27e6b0f4/xlwt-1.3.0-py2.py3-none-any.whl
Collecting xlrd (from tablib->django-import-export)
Using cached https://files.pythonhosted.org/packages/b0/16/63576a1a001752e34bf8ea62e367997530dc553b689356b9879339cf45a4/xlrd-1.2.0-py2.py3-none-any.whl
Collecting backports.csv (from tablib->django-import-export)
Using cached https://files.pythonhosted.org/packages/8e/26/a6bd68f13e0f38fbb643d6e497fc3462be83a0b6c4d43425c78bb51a7291/backports.csv-1.0.7-py2.py3-none-any.whl
Collecting odfpy (from tablib->django-import-export)
Using cached https://files.pythonhosted.org/packages/85/7d/8f6d1f2a4683be362b101c00232b4c3839e4e4a90e0945d8d43ec6aa671d/odfpy-1.4.0.tar.gz
Collecting pyyaml (from tablib->django-import-export)
Using cached https://files.pythonhosted.org/packages/e3/e8/b3212641ee2718d556df0f23f78de8303f068fe29cdaa7a91018849582fe/PyYAML-5.1.2.tar.gz
Requirement already satisfied: sqlparse in ./.env/lib/python3.7/site-packages (from django>=1.8->django-import-export) (0.3.0)
Requirement already satisfied: pytz in ./.env/lib/python3.7/site-packages (from django>=1.8->django-import-export) (2019.2)
Collecting jdcal (from openpyxl>=2.4.0->tablib->django-import-export)
Using cached https://files.pythonhosted.org/packages/f0/da/572cbc0bc582390480bbd7c4e93d14dc46079778ed915b505dc494b37c57/jdcal-1.4.1-py2.py3-none-any.whl
Collecting et_xmlfile (from openpyxl>=2.4.0->tablib->django-import-export)
Using cached https://files.pythonhosted.org/packages/22/28/a99c42aea746e18382ad9fb36f64c1c1f04216f41797f2f0fa567da11388/et_xmlfile-1.0.1.tar.gz
Collecting defusedxml (from odfpy->tablib->django-import-export)
Using cached https://files.pythonhosted.org/packages/06/74/9b387472866358ebc08732de3da6dc48e44b0aacd2ddaa5cb85ab7e986a2/defusedxml-0.6.0-py2.py3-none-any.whl
Installing collected packages: jdcal, et-xmlfile, openpyxl, xlwt, xlrd, backports.csv, defusedxml, odfpy, pyyaml, tablib, diff-match-patch, django-import-export
Running setup.py install for et-xmlfile ... done
Running setup.py install for openpyxl ... done
Running setup.py install for odfpy ... done
Running setup.py install for pyyaml ... done
Running setup.py install for diff-match-patch ... done
Successfully installed backports.csv-1.0.7 defusedxml-0.6.0 diff-match-patch-20181111 django-import-export-1.2.0 et-xmlfile-1.0.1 jdcal-1.4.1 odfpy-1.4.0 openpyxl-2.6.3 pyyaml-5.1.2 tablib-0.13.0 xlrd-1.2.0 xlwt-1.3.0
WARNING: You are using pip version 19.2.2, however version 19.2.3 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
Python version:
(.env) [travis#spooky issuetracker]$ python -V
Python 3.7.4
Pip version:
(.env) [travis#spooky issuetracker]$ pip -V
pip 19.2.2 from /home/travis/Documents/Python/Django/Projects/issuetracker/.env/lib/python3.7/site-packages/pip (python 3.7)
.env/bin/pip:
#!/home/travis/Documents/Python/Django/Projects/issuetracker/.env/bin/python
# -*- coding: utf-8 -*-
import re
import sys
from pip._internal import main
if __name__ == '__main__':
sys.argv[0] = re.sub(r'(-script\.pyw?|\.exe)?$', '', sys.argv[0])
sys.exit(main())
I'm not sure when this stopped working, since I was previously able to successfully install other packages into the same virtual environment. How can I ensure that pip is installing packages correctly into my virtual environment?
Update:
It seems that pip is installing into the .local/lib/python3.7/site-packages directory instead of .env/lib/python3.7/site-packages.

Found a fix/workaround in the answer to this question:
Pip installing packages to global site-packages when inside virtual environment
Like the OP of that question, I'm also running Manjaro Linux with the latest updates.

from google.cloud import language ModuleNotFoundError: No module named 'google.cloud'

I had a code that I was able to run last year using Google Cloud Natural Language. I am not able to this year due to a library problem. I used the following methods to install it. However, not successful. How should install it?
[jalal#goku sentiment]$ pip install --user --upgrade google-cloud
Collecting google-cloud
Downloading https://files.pythonhosted.org/packages/ba/b1/7c54d1950e7808df06642274e677dbcedba57f75307adf2e5ad8d39e5e0e/google_cloud-0.34.0-py2.py3-none-any.whl
Installing collected packages: google-cloud
Successfully installed google-cloud-0.34.0
[jalal#goku sentiment]$ python sentiment.py
Traceback (most recent call last):
File "sentiment.py", line 20, in <module>
from google.cloud import language
ModuleNotFoundError: No module named 'google.cloud'
[jalal#goku sentiment]$ python -m pip install google-cloud
Requirement already satisfied: google-cloud in /home/grad3/jalal/.local/lib/python3.6/site-packages (0.34.0)
The error is:
$ python sentiment.py
Traceback (most recent call last):
File "sentiment.py", line 20, in <module>
from google.cloud import language
ModuleNotFoundError: No module named 'google.cloud'
A little about my system setup:
$ which python
/scratch/sjn-p3/anaconda/anaconda3/bin/python
$ python
Python 3.6.4 |Anaconda custom (64-bit)| (default, Jan 16 2018, 18:10:19)
[GCC 7.2.0] on linux
$ lsb_release -a
LSB Version: :core-4.1-amd64:core-4.1-noarch
Distributor ID: CentOS
Description: CentOS Linux release 7.6.1810 (Core)
Release: 7.6.1810
Codename: Core
Also, I find it very weird when I am installing the pip package with Anacoda pip why does it install it into native Python and not Anaconda Python?
$ /scratch/sjn-p3/anaconda/anaconda3/bin/pip install --user --upgrade google-cloud
Requirement already up-to-date: google-cloud in /home/grad3/jalal/.local/lib/python3.6/site-packages (0.34.0)

The issue is you are using the --user flag and pip but wish to run the Anaconda distribution of Python.
If you do python -c 'import site; print(site.USER_BASE)' from the CLI it will return /home/grad3/jalal/.local hence why packages are installed there when you pass the --user flag to pip. Since you want to use the Anaconda distribution you should stay consistent and use their version of a package manager, which would be conda in this case, which you can find about more --> here.
However, I highly recommend using virtual environments so that this issue can be avoided in the future. You can find out more about Anaconda virtual environments over --> here.
If you wish to keep using pip along with Anaconda you can modify your target directory in the ~/.pip/pip.conf file to point at the location you want. Probably something along these lines will work:
[global]
target=/scratch/sjn-p3/anaconda/anaconda3/lib/python{version}/site-packages

Solved the problem by the following command. You have to install the specific product from the Google Cloud (in this case Natural Language):
$ pip install --user --upgrade google-cloud-language
Collecting google-cloud-language
Using cached https://files.pythonhosted.org/packages/b1/3a/6f5808421aea97363cb19a095f73ed6eed12bbaee1e67f2145af7125585d/google_cloud_language-1.1.1-py2.py3-none-any.whl
Collecting google-api-core[grpc]<2.0.0dev,>=1.6.0 (from google-cloud-language)
Using cached https://files.pythonhosted.org/packages/7d/73/e4877e921fe59307ec6b1b0b0c2ad9fde2d1c6bab8dd06ec913891a20dc6/google_api_core-1.8.2-py2.py3-none-any.whl
Requirement already satisfied, skipping upgrade: protobuf>=3.4.0 in /scratch/sjn-p3/anaconda/anaconda3/lib/python3.6/site-packages (from google-api-core[grpc]<2.0.0dev,>=1.6.0->google-cloud-language) (3.6.1)
Requirement already satisfied, skipping upgrade: pytz in /scratch/sjn-p3/anaconda/anaconda3/lib/python3.6/site-packages (from google-api-core[grpc]<2.0.0dev,>=1.6.0->google-cloud-language) (2017.3)
Requirement already satisfied, skipping upgrade: six>=1.10.0 in /scratch/sjn-p3/anaconda/anaconda3/lib/python3.6/site-packages (from google-api-core[grpc]<2.0.0dev,>=1.6.0->google-cloud-language) (1.11.0)
Requirement already satisfied, skipping upgrade: requests<3.0.0dev,>=2.18.0 in /home/grad3/jalal/.local/lib/python3.6/site-packages (from google-api-core[grpc]<2.0.0dev,>=1.6.0->google-cloud-language) (2.21.0)
Collecting google-auth<2.0dev,>=0.4.0 (from google-api-core[grpc]<2.0.0dev,>=1.6.0->google-cloud-language)
Using cached https://files.pythonhosted.org/packages/c5/9b/ed0516cc1f7609fb0217e3057ff4f0f9f3e3ce79a369c6af4a6c5ca25664/google_auth-1.6.3-py2.py3-none-any.whl
Collecting googleapis-common-protos!=1.5.4,<2.0dev,>=1.5.3 (from google-api-core[grpc]<2.0.0dev,>=1.6.0->google-cloud-language)
Requirement already satisfied, skipping upgrade: setuptools>=34.0.0 in /scratch/sjn-p3/anaconda/anaconda3/lib/python3.6/site-packages (from google-api-core[grpc]<2.0.0dev,>=1.6.0->google-cloud-language) (38.4.0)
Requirement already satisfied, skipping upgrade: grpcio>=1.8.2; extra == "grpc" in /scratch/sjn-p3/anaconda/anaconda3/lib/python3.6/site-packages (from google-api-core[grpc]<2.0.0dev,>=1.6.0->google-cloud-language) (1.15.0)
Requirement already satisfied, skipping upgrade: idna<2.9,>=2.5 in /scratch/sjn-p3/anaconda/anaconda3/lib/python3.6/site-packages (from requests<3.0.0dev,>=2.18.0->google-api-core[grpc]<2.0.0dev,>=1.6.0->google-cloud-language) (2.6)
Requirement already satisfied, skipping upgrade: urllib3<1.25,>=1.21.1 in /scratch/sjn-p3/anaconda/anaconda3/lib/python3.6/site-packages (from requests<3.0.0dev,>=2.18.0->google-api-core[grpc]<2.0.0dev,>=1.6.0->google-cloud-language) (1.22)
Requirement already satisfied, skipping upgrade: certifi>=2017.4.17 in /scratch/sjn-p3/anaconda/anaconda3/lib/python3.6/site-packages (from requests<3.0.0dev,>=2.18.0->google-api-core[grpc]<2.0.0dev,>=1.6.0->google-cloud-language) (2018.10.15)
Requirement already satisfied, skipping upgrade: chardet<3.1.0,>=3.0.2 in /scratch/sjn-p3/anaconda/anaconda3/lib/python3.6/site-packages (from requests<3.0.0dev,>=2.18.0->google-api-core[grpc]<2.0.0dev,>=1.6.0->google-cloud-language) (3.0.4)
Collecting pyasn1-modules>=0.2.1 (from google-auth<2.0dev,>=0.4.0->google-api-core[grpc]<2.0.0dev,>=1.6.0->google-cloud-language)
Using cached https://files.pythonhosted.org/packages/da/98/8ddd9fa4d84065926832bcf2255a2b69f1d03330aa4d1c49cc7317ac888e/pyasn1_modules-0.2.4-py2.py3-none-any.whl
Collecting cachetools>=2.0.0 (from google-auth<2.0dev,>=0.4.0->google-api-core[grpc]<2.0.0dev,>=1.6.0->google-cloud-language)
Using cached https://files.pythonhosted.org/packages/39/2b/d87fc2369242bd743883232c463f28205902b8579cb68dcf5b11eee1652f/cachetools-3.1.0-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from google-auth<2.0dev,>=0.4.0->google-api-core[grpc]<2.0.0dev,>=1.6.0->google-cloud-language)
Using cached https://files.pythonhosted.org/packages/02/e5/38518af393f7c214357079ce67a317307936896e961e35450b70fad2a9cf/rsa-4.0-py2.py3-none-any.whl
Collecting pyasn1<0.5.0,>=0.4.1 (from pyasn1-modules>=0.2.1->google-auth<2.0dev,>=0.4.0->google-api-core[grpc]<2.0.0dev,>=1.6.0->google-cloud-language)
Using cached https://files.pythonhosted.org/packages/7b/7c/c9386b82a25115cccf1903441bba3cbadcfae7b678a20167347fa8ded34c/pyasn1-0.4.5-py2.py3-none-any.whl
Installing collected packages: pyasn1, pyasn1-modules, cachetools, rsa, google-auth, googleapis-common-protos, google-api-core, google-cloud-language
The scripts pyrsa-decrypt, pyrsa-encrypt, pyrsa-keygen, pyrsa-priv2pub, pyrsa-sign and pyrsa-verify are installed in '/home/grad3/jalal/.local/bin' which is not on PATH.
Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
Successfully installed cachetools-3.1.0 google-api-core-1.8.2 google-auth-1.6.3 google-cloud-language-1.1.1 googleapis-common-protos-1.5.9 pyasn1-0.4.5 pyasn1-modules-0.2.4 rsa-4.0
$ vi ~/.bashrc
Added this line to the end of my ~/.bashrc (or whatever it is told to you):
export PATH=/home/grad3/jalal/.local/bin:$PATH
$ source ~/.bashrc