Develop Reference

How do I make sure conda's environment-specific paths are at the top of sys.path?

I have a conda environment, myenv. I realized that the Python interpreter in this environment tries to import packages from /Users/me/.local/lib/python3.8/site-packages before /usr/local/anaconda3/envs/myenv/lib/python3.8/site-packages. I would have expected all the environment-specific paths to be appended to the beginning of sys.path. Is this expected behavior? In this case, I want to import conda-installed versions of numpy, scipy and numexpr since they use the Intel MKL backend.
(myenv) me$ which python
/usr/local/anaconda3/envs/myenv/bin/python
(myenv) me$ python
Python 3.8.5 (default, Sep 4 2020, 02:22:02)
[Clang 10.0.0 ] :: Anaconda, Inc. on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy as np
>>> np.__file__
>>> '/Users/me/.local/lib/python3.8/site-packages/numpy/__init__.py'
>>> import sys
>>> # /Users/me/dir* refer to directories that contain local modules I've pip installed
>>> print("\n".join(sys.path))
/usr/local/anaconda3/envs/myenv/lib/python38.zip
/usr/local/anaconda3/envs/myenv/lib/python3.8
/usr/local/anaconda3/envs/myenv/lib/python3.8/lib-dynload
/Users/me/.local/lib/python3.8/site-packages
/Users/me/dir1
/Users/me/dir2
/Users/me/dir3
/usr/local/anaconda3/envs/myenv/lib/python3.8/site-packages
Other things that might be relevant:
I use conda and pyenv side-by-side so have conda's auto_activate_base setting set to false (as suggested here).
The Python interpreter declares that it's version 3.8.5, but conda info lists python version as 3.8.3.final.0.
I'm using conda version 4.9.2 on a MacOS Version 11.2.2.

Thanks to #merv's comment, I realized that the issue was that pip's user-site is prioritized above the conda environment's package directory. I resolved this as #merv suggested: by uninstalling all the packages in pip's user-site.
me$ pyenv shell 3.8.1
me$ pip freeze --user | grep -Eo "[[:alnum:]\-]+==[[:digit:]]{1,2}\.[[:digit:]]{1,2}(\.[[:digit:]]{1,2})?" | xargs pip uninstall -y
me$ conda activate myenv
(myenv) me$ python
Python 3.8.5 (default, Sep 4 2020, 02:22:02)
[Clang 10.0.0 ] :: Anaconda, Inc. on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy as np
>>> np.__file__
/usr/local/anaconda3/envs/myenv/lib/python3.8/site-packages/numpy/__init__.py

Something wrong with my conda channel and leads to a tensorflow import error

I created a Conda environment with python 3.6 and venv and activated it. then installed tensorflow with
pip install tensorflow
However, there is something wrong with my python. Basically like when i run python i get:
Python 3.6.2 |Anaconda, Inc.| (default, Oct 5 2017, 07:59:26) [GCC 7.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import tensorflow as tf
>>> Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'tensorflow'
but with python3.6 everything works fine:
Python 3.6.10 | packaged by conda-forge | (default, Apr 24 2020, 16:44:11) [GCC 7.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import tensorflow as tf
>>>
I basically want to type the "python" into my command line and to work fine with tensorFlow.

Something is really strange with the way you are doing this, you shouldn't need a venv on top of your environment and you should install tensorflow with a conda rather than a pip install (reason being conda is an environment manager and solves for all packages to work together, whereas pip may install or uninstall things in a way that messes up the whole environment solution; tldr, more likely to work). Try making a new environment and see how this works out
conda create -n tfenv tensorflow=2
if you instead want Python 3.6 (other one will use latest python that works with tensorflow, should be 3.7)
conda create -n tfenv python=3.6 tensorflow=2
then, as usual activate the environment with
conda activate tfenv

PyDev import error even though module is installed

I'm trying to use the module pyelastix in PyDev but the line import pyelastix keeps yielding ImportError: No module named 'pyelastix'. I'm using a virtual environment called cv on Ubuntu 16.04. I tried the following commands to install pyelastix:
sudo pip install pyelastix
sudo pip3 install pyelastix
inside and outside of the virtual environment. Importing it in the CLI works fine:
Python 3.5.2 (default, Nov 23 2017, 16:37:01)
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pyelastix
>>>
In Eclipse, the PyDev Interpreter is set to /home/username/.virtualenvs/cv/bin/python3.5. The PYTHONPATH source folder is /${PROJECT_DIR_NAME}. What could be the issue here?

Build Python as UCS-4 via pyenv

I run into this issue ImportError numpy/core/multiarray.so: undefined symbol: PyUnicodeUCS2_AsASCIIString installing Python in a pyenv-virtualenv environment.
In my case, it happens with the matplotlib package instead of numpy (as in the above question), but it's basically the same issue.
The answer given in that question is a simple:
Rebuild NumPy against a Python built as UCS-4.
I don't know how to do this. In this other question it is said that one has to use:
./configure --enable-unicode=ucs4
but I don't know how to use that command along with pyenv.
This issue is also mentioned in pyenv's repo issue list, and a solution given in a comment. Sadly (for me) I can not understand how to apply the fix explained in said comment.
So my question basically is: how do I build Python as UCS-4 via pyenv?

Installing python with pyenv with ucs2:
$ export PYTHON_CONFIGURE_OPTS=--enable-unicode=ucs2
$ pyenv install -v 2.7.11
...
$ pyenv local 2.7.11
$ pyenv versions
system
* 2.7.11 (set by /home/nwani/.python-version)
$ /home/nwani/.pyenv/shims/python
Python 2.7.11 (default, Aug 13 2016, 13:42:13)
[GCC 4.8.4] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sysconfig
>>> sysconfig.get_config_vars()['CONFIG_ARGS']
"'--prefix=/home/nwani/.pyenv/versions/2.7.11' '--enable-unicode=ucs2' '--libdir=/home/nwani/.pyenv/versions/2.7.11/lib' 'LDFLAGS=-L/home/nwani/.pyenv/versions/2.7.11/lib ' 'CPPFLAGS=-I/home/nwani/.pyenv/versions/2.7.11/include '"
Installing python with pyenv with ucs4:
$ pyenv uninstall 2.7.11
pyenv: remove /home/nwani/.pyenv/versions/2.7.11? y
$ export PYTHON_CONFIGURE_OPTS=--enable-unicode=ucs4
$ pyenv install -v 2.7.11
...
$ pyenv local 2.7.11
$ pyenv versions
system
* 2.7.11 (set by /home/nwani/.python-version)
$ /home/nwani/.pyenv/shims/python
Python 2.7.11 (default, Aug 13 2016, 13:49:09)
[GCC 4.8.4] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sysconfig
>>> sysconfig.get_config_vars()['CONFIG_ARGS']
"'--prefix=/home/nwani/.pyenv/versions/2.7.11' '--enable-unicode=ucs4' '--libdir=/home/nwani/.pyenv/versions/2.7.11/lib' 'LDFLAGS=-L/home/nwani/.pyenv/versions/2.7.11/lib ' 'CPPFLAGS=-I/home/nwani/.pyenv/versions/2.7.11/include '"

Installing Scipy in Virtualenv

I've tried many ways to install scipy but without successes. I use virtualenv on Linux Mate.
After the workon command I tried
pip install scipy
or
easy_install scipy
end every time is a "failed with exit status 1"
I tried also
apt-get install python-scipy
and I've no errors but if I run python
python
import scipy
I'll have "ImportError: No module named scipy"
What is wrong?

I'd suggest using Miniconda to install scipy.
wget https://repo.continuum.io/miniconda/Miniconda-latest-Linux-x86_64.sh
bash Miniconda-latest-Linux-x86_64.sh -b
Then, create a Conda environment with scipy installed:
conda create -n scipy scipy
Now you'll have access to the activate and deactivate scripts, used to open and close Conda environments.
$ source activate scipy
discarding /Users/username/miniconda/bin from PATH
prepending /Users/username/miniconda/envs/scipy/bin to PATH
(scipy)$ python
Python 2.7.11 |Continuum Analytics, Inc.| (default, Dec 6 2015, 18:57:58)
[GCC 4.2.1 (Apple Inc. build 5577)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
Anaconda is brought to you by Continuum Analytics.
Please check out: http://continuum.io/thanks and https://anaconda.org
>>> import scipy
>>> scipy.version.version
'0.16.0'
[Edit] You indicated you want to use the system installed scipy package, that you have successfully installed via apt, in a virtualenv. You can instruct virtualenv to create a new environment that includes all the system Python packages like this:
$ virtualenv --system-site-packages scipy_env
New python executable in scipy_env/bin/python
Installing setuptools, pip...done.
$ source scipy_env/bin/activate
(scipy_env)$ python
Python 2.7.10 (default, Jul 14 2015, 19:46:27)
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.39)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import scipy