Develop Reference

How do I make sure conda's environment-specific paths are at the top of sys.path?

I have a conda environment, myenv. I realized that the Python interpreter in this environment tries to import packages from /Users/me/.local/lib/python3.8/site-packages before /usr/local/anaconda3/envs/myenv/lib/python3.8/site-packages. I would have expected all the environment-specific paths to be appended to the beginning of sys.path. Is this expected behavior? In this case, I want to import conda-installed versions of numpy, scipy and numexpr since they use the Intel MKL backend.
(myenv) me$ which python
/usr/local/anaconda3/envs/myenv/bin/python
(myenv) me$ python
Python 3.8.5 (default, Sep 4 2020, 02:22:02)
[Clang 10.0.0 ] :: Anaconda, Inc. on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy as np
>>> np.__file__
>>> '/Users/me/.local/lib/python3.8/site-packages/numpy/__init__.py'
>>> import sys
>>> # /Users/me/dir* refer to directories that contain local modules I've pip installed
>>> print("\n".join(sys.path))
/usr/local/anaconda3/envs/myenv/lib/python38.zip
/usr/local/anaconda3/envs/myenv/lib/python3.8
/usr/local/anaconda3/envs/myenv/lib/python3.8/lib-dynload
/Users/me/.local/lib/python3.8/site-packages
/Users/me/dir1
/Users/me/dir2
/Users/me/dir3
/usr/local/anaconda3/envs/myenv/lib/python3.8/site-packages
Other things that might be relevant:
I use conda and pyenv side-by-side so have conda's auto_activate_base setting set to false (as suggested here).
The Python interpreter declares that it's version 3.8.5, but conda info lists python version as 3.8.3.final.0.
I'm using conda version 4.9.2 on a MacOS Version 11.2.2.

Thanks to #merv's comment, I realized that the issue was that pip's user-site is prioritized above the conda environment's package directory. I resolved this as #merv suggested: by uninstalling all the packages in pip's user-site.
me$ pyenv shell 3.8.1
me$ pip freeze --user | grep -Eo "[[:alnum:]\-]+==[[:digit:]]{1,2}\.[[:digit:]]{1,2}(\.[[:digit:]]{1,2})?" | xargs pip uninstall -y
me$ conda activate myenv
(myenv) me$ python
Python 3.8.5 (default, Sep 4 2020, 02:22:02)
[Clang 10.0.0 ] :: Anaconda, Inc. on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy as np
>>> np.__file__
/usr/local/anaconda3/envs/myenv/lib/python3.8/site-packages/numpy/__init__.py

How to fix PYTHONPATHS and a weird error of Python?

Yesterday, I did put my laptop on upgrade 19.10 to 20.04 but due to power failure, that became a partial-upgrade, the system broked. I resolved everything but my Django app wasn't running due to PYTHONPPATH so I tried uninstalling python3 and everything got broken. I re-installed that again.
Now when I do python --version I got
bash: python: command not found
whereas python3 --version gives correct answer.
Python 3.8.2
I have python2.7 and python3 both installed. So for now, my Python is not working and also I think I've messed up my PYTHONPATH and I really don't know what I'm going to do now.
My ./~bashrc file looks like below :
# Install Ruby Gems to ~/gems
export GEM_HOME=$HOME/gems
export PATH=$HOME/gems/bin:$PATH
# Install Ruby Gems to ~/gems
export GEM_HOME=$HOME/gems
export PATH=$HOME/gems/bin:$PATH
# Install Ruby Gems to ~/gems
export GEM_HOME="$HOME/gems"
export PATH="$HOME/gems/bin:$PATH"
I'm using Ubuntu 20.04.

Please specify how are you running your project and what exactly is the issue you are facing. May be you can paste the error message you get.
For python command,
In Linux, generally the base commands (like python) without version in it, would actually be pointing the specific (python) version executable through symbolic links (or simply links).
[foo#linuxbox ~]$ ls -l /usr/bin/python
lrwxrwxrwx. 1 root root 16 Feb 9 16:26 /usr/bin/python -> /usr/bin/python3
These links can be created or even edited to our need to point to the version we need. Use the below command to link python to python3. This is equivalent to setting alias for python3 as python but bit more than that as all users/process can run python but in case of alias the tool/user must be running from bash or corresponding shell where alias was created.
sudo ln -f -s /usr/bin/python3 /usr/bin/python

I feel in Ubuntu 20 you have to run command python2 to go into 2.7.* interpreter. python and python3 command both refers to Python3. But anyway your python command should work.
#ideapad:~$ python
Python 3.6.9 (default, Apr 18 2020, 01:56:04)
[GCC 8.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> exit()
ideapad:~$ python2
Python 2.7.17 (default, Apr 15 2020, 17:20:14)
[GCC 7.5.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> exit()
ideapad:~$ python3
Python 3.6.9 (default, Apr 18 2020, 01:56:04)
[GCC 8.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>
To solve your issue, use an alias. Place command alias python=python3 into ~/.bashrc file, after adding this run source ~/.bashrc.
Other solutions:
run command which python it will reveal the location of installed Python and then try adding the location given by which python command to PYTHONPATH
Reinstall your python - sudo apt install python

pyenv, virtualenv, autoenv can not recognize python

I'm using pyenv, virtualenv, autoenv.
When I activated pyenv, it shows an error.
Chois#Chois-MacPro ~ $pyenv activate chacha_dabang
pyenv-virtualenv: prompt changing will be removed from future release. configure `export PYENV_VIRTUALENV_DISABLE_PROMPT=1' to simulate the behavior.
(chacha_dabang) Chois#Chois-MacPro ~ $python
pyenv: python3.5: command not found
The `python3.5' command exists in these Python versions:
3.5.1
So, I checked with which:
(chacha_dabang) Chois#Chois-MacPro ~ $which python
/Users/Chois/.pyenv/shims/python
I execute python with this one :
(chacha_dabang) Chois#Chois-MacPro ~ $/Users/Chois/.pyenv/shims/python
Python 3.5.1 (default, Aug 3 2016, 11:09:57)
[GCC 4.2.1 Compatible Apple LLVM 7.3.0 (clang-703.0.31)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>>
It works well!!!
I don't know why it doesn't work when I type just python
Need your help.
Here is my PATH :
Chois#Chois-MacPro ~ $echo $PATH
/Usr/local/Cellar/pyenv-virtualenv/20160716/shims:/Users/Chois/.pyenv/shims:/opt/local/bin:/opt/local/sbin:/Library/Frameworks/Python.framework/Versions/3.5/bin:/Users/Chois/Library:/usr/local/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin:/usr/local/git/bin:/Users/Chois/phantomjs-1.9.2-macosx/bin
and .pyenv:
Chois#Chois-MacPro versions $pwd
/Users/Chois/.pyenv/versions
Chois#Chois-MacPro versions $ls
3.5.1 chacha_dabang

Installing Scipy in Virtualenv

I've tried many ways to install scipy but without successes. I use virtualenv on Linux Mate.
After the workon command I tried
pip install scipy
or
easy_install scipy
end every time is a "failed with exit status 1"
I tried also
apt-get install python-scipy
and I've no errors but if I run python
python
import scipy
I'll have "ImportError: No module named scipy"
What is wrong?

I'd suggest using Miniconda to install scipy.
wget https://repo.continuum.io/miniconda/Miniconda-latest-Linux-x86_64.sh
bash Miniconda-latest-Linux-x86_64.sh -b
Then, create a Conda environment with scipy installed:
conda create -n scipy scipy
Now you'll have access to the activate and deactivate scripts, used to open and close Conda environments.
$ source activate scipy
discarding /Users/username/miniconda/bin from PATH
prepending /Users/username/miniconda/envs/scipy/bin to PATH
(scipy)$ python
Python 2.7.11 |Continuum Analytics, Inc.| (default, Dec 6 2015, 18:57:58)
[GCC 4.2.1 (Apple Inc. build 5577)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
Anaconda is brought to you by Continuum Analytics.
Please check out: http://continuum.io/thanks and https://anaconda.org
>>> import scipy
>>> scipy.version.version
'0.16.0'
[Edit] You indicated you want to use the system installed scipy package, that you have successfully installed via apt, in a virtualenv. You can instruct virtualenv to create a new environment that includes all the system Python packages like this:
$ virtualenv --system-site-packages scipy_env
New python executable in scipy_env/bin/python
Installing setuptools, pip...done.
$ source scipy_env/bin/activate
(scipy_env)$ python
Python 2.7.10 (default, Jul 14 2015, 19:46:27)
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.39)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import scipy

can't detect default python version

~$ python
bash: python: command not found
while running python2 and python 3 its showing different versions.
~$ python2
Python 2.7.6 (default, Jun 22 2015, 17:58:13)
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
~$ python3
Python 3.4.0 (default, Jun 19 2015, 14:20:21)
[GCC 4.8.2] on linux
Type "help", "copyright", "credits" or "license" for more information.
i think somehow default python link is broken. i can't seem to access ubuntu software center also.
Please suggest a way to fix this.

Rather than re-creating the simlink as suggested, I would start working with virtualenv. You can create virtual environment pointing at different python versions by doing:
virtualenv -p /usr/bin/python2 venv_name (for Python 2.7)
virtualenv -p /usr/bin/python3 venv_name (for Python 3)
Once you go into the venv and do the source venv_name/bin/activate, your python symbolic link will be pointed at the version you've chosen when creating it.
Also, you might find a pyvenv link which will be associated to Python 3 as this version of Python comes with virtualenv (and pip) out of the box. You might need to install virtualenv with pip (or easy install), and to figure it out which Pythonversion is your pip associated to, you can do pip --version