I have different anaconda environments. When starting up one of them it seems as if sys.path is still set to some totally wrong directory. Am I not understanding the concepts of environments correctly or is this an error concerning my anaconda setup?
My environments:
fabianwerner ~$ conda env list
# conda environments:
#
base * /Users/fabianwerner/anaconda3/anaconda3
my-rdkit-env /Users/fabianwerner/anaconda3/anaconda3/envs/my-rdkit-env
openpose /Users/fabianwerner/anaconda3/anaconda3/envs/openpose
fabianwerner ~$
I activate the environment 'openpose':
fabianwerner ~$ conda activate openpose
(openpose) fabianwerner ~$
I will try to import 'keras' so let us make sure that it is installed:
(openpose) fabianwerner ~$ pip list | grep -i keras
DEPRECATION: The default format will switch to columns in the future. You can use --format=(legacy|columns) (or define a format=(legacy|columns) in your pip.conf under the [list] section) to disable this warning.
Keras (2.2.4)
Keras-Applications (1.0.7)
Keras-Preprocessing (1.0.9)
You are using pip version 9.0.1, however version 19.1.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
(openpose) fabianwerner ~$
I start python and try to import keras:
(openpose) fabianwerner ~$ python
Python 3.6.5 |Anaconda, Inc.| (default, Apr 26 2018, 08:42:37)
[GCC 4.2.1 Compatible Clang 4.0.1 (tags/RELEASE_401/final)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import keras
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'keras'
>>>
Let us check where python looks for modules:
>>> import sys
>>> print(sys.path)
['', '/Users/fabianwerner/anaconda3/anaconda3/envs/my-rdkit-env/lib/python36.zip', '/Users/fabianwerner/anaconda3/anaconda3/envs/my-rdkit-env/lib/python3.6', '/Users/fabianwerner/anaconda3/anaconda3/envs/my-rdkit-env/lib/python3.6/lib-dynload', '/Users/fabianwerner/.local/lib/python3.6/site-packages', '/Users/fabianwerner/anaconda3/anaconda3/envs/my-rdkit-env/lib/python3.6/site-packages']
>>>
so I activated the environment 'openpose' but it still looks for modules in the other (wrong!) environment 'my-rdkit-env'!
How to fix this?
EDIT:
I cannot see any obvious problem with the anaconda setup as conda info -a gives the right path:
(openpose) fabianwerner ~$ conda info -a
active environment : openpose
active env location : /Users/fabianwerner/anaconda3/anaconda3/envs/openpose
shell level : 2
user config file : /Users/fabianwerner/.condarc
populated config files :
conda version : 4.5.11
conda-build version : 3.15.1
python version : 3.7.0.final.0
base environment : /Users/fabianwerner/anaconda3/anaconda3 (writable)
channel URLs : https://repo.anaconda.com/pkgs/main/osx-64
https://repo.anaconda.com/pkgs/main/noarch
https://repo.anaconda.com/pkgs/free/osx-64
https://repo.anaconda.com/pkgs/free/noarch
https://repo.anaconda.com/pkgs/r/osx-64
https://repo.anaconda.com/pkgs/r/noarch
https://repo.anaconda.com/pkgs/pro/osx-64
https://repo.anaconda.com/pkgs/pro/noarch
package cache : /Users/fabianwerner/anaconda3/anaconda3/pkgs
/Users/fabianwerner/.conda/pkgs
envs directories : /Users/fabianwerner/anaconda3/anaconda3/envs
/Users/fabianwerner/.conda/envs
platform : osx-64
user-agent : conda/4.5.11 requests/2.19.1 CPython/3.7.0 Darwin/17.7.0 OSX/10.13.6
UID:GID : 502:20
netrc file : None
offline mode : False
# conda environments:
#
base /Users/fabianwerner/anaconda3/anaconda3
my-rdkit-env /Users/fabianwerner/anaconda3/anaconda3/envs/my-rdkit-env
openpose * /Users/fabianwerner/anaconda3/anaconda3/envs/openpose
strokinat0r /Users/fabianwerner/anaconda3/anaconda3/envs/strokinat0r
sys.version: 3.7.0 (default, Jun 28 2018, 07:39:16)
...
sys.prefix: /Users/fabianwerner/anaconda3/anaconda3
sys.executable: /Users/fabianwerner/anaconda3/anaconda3/bin/python
conda location: /Users/fabianwerner/anaconda3/anaconda3/lib/python3.7/site-packages/conda
conda-build: /Users/fabianwerner/anaconda3/anaconda3/bin/conda-build
conda-convert: /Users/fabianwerner/anaconda3/anaconda3/bin/conda-convert
conda-develop: /Users/fabianwerner/anaconda3/anaconda3/bin/conda-develop
conda-env: /Users/fabianwerner/anaconda3/anaconda3/bin/conda-env
conda-index: /Users/fabianwerner/anaconda3/anaconda3/bin/conda-index
conda-inspect: /Users/fabianwerner/anaconda3/anaconda3/bin/conda-inspect
conda-metapackage: /Users/fabianwerner/anaconda3/anaconda3/bin/conda-metapackage
conda-render: /Users/fabianwerner/anaconda3/anaconda3/bin/conda-render
conda-server: /Users/fabianwerner/anaconda3/anaconda3/bin/conda-server
conda-skeleton: /Users/fabianwerner/anaconda3/anaconda3/bin/conda-skeleton
user site dirs: ~/.local/lib/python3.6
CIO_TEST: <not set>
CONDA_DEFAULT_ENV: openpose
CONDA_EXE: /Users/fabianwerner/anaconda3/anaconda3/bin/conda
CONDA_PREFIX: /Users/fabianwerner/anaconda3/anaconda3/envs/openpose
CONDA_PREFIX_1: /Users/fabianwerner/anaconda3/anaconda3
CONDA_PROMPT_MODIFIER: (openpose)
CONDA_PYTHON_EXE: /Users/fabianwerner/anaconda3/anaconda3/bin/python
CONDA_ROOT: /Users/fabianwerner/anaconda3/anaconda3
CONDA_SHLVL: 2
PATH: /Users/fabianwerner/anaconda3/anaconda3/envs/openpose/bin:/Users/fabianwerner/anaconda3/anaconda3/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Applications/GIMP-2.10.app/Contents/MacOS
REQUESTS_CA_BUNDLE: <not set>
SSL_CERT_FILE: <not set>
WARNING: could not import _license.show_info
# try:
# $ conda install -n root _license
(openpose) fabianwerner ~$
Also, I do not seem to have an environment variable set that relates to python:
(openpose) fabianwerner ~$ printenv | grep -i python
CONDA_PYTHON_EXE=/Users/fabianwerner/anaconda3/anaconda3/bin/python
(openpose) fabianwerner ~$
My setup:
Mac OS-X 10.13.6 (High Sierra)
anaconda version 4.5.11
Thanks,
FW
The Python interpreter you started in your example is not the one in the environment.
conda info -a says python version : 3.7.0.final.0 and yet your interpreter says Python 3.6.5
The problem should become apparent when you activate your environment and run which python which should be pointing to the activated env but probably doesn't.
How did you create those environments? Make sure to set the python=XX option or the new environment uses the interpreter from the base/root environment rather than installing a new one. I.e. conda create -n my_environment python=3.7
Edit:
Sorry, I just looked up and tested conda info -a. python version : XX seems to be referring to the base env not the currently active one.
I'm leaving this answer here, since even though my reasoning seems to be wrong, it may still be helpful.
Related
I'm on Ubuntu 20.04 LTS with CUDA 11.1 installed (and working, with PATH and LD_LIBRARY_PATH configured correctly), and I'm trying to define a reusable conda environment (i.e., in an environment.yml file) that successfully installs PyTorch with CUDA support.
However, when I use the environment file, I get a message that Torch wasn't compiled with CUDA support:
Python 3.8.10 | packaged by conda-forge | (default, May 11 2021, 07:01:05)
[GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import torch
>>> device = torch.device("cuda:0")
>>> t = torch.tensor(device=device, data=[0,1,2,3])
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/jdr2160/anaconda3/envs/foo/lib/python3.8/site-packages/torch/cuda/__init__.py", line 166, in _lazy_init
raise AssertionError("Torch not compiled with CUDA enabled")
AssertionError: Torch not compiled with CUDA enabled
My environment.yml is pretty bare-bones:
name: foo
channels:
- conda-forge
- nvidia
- pytorch
dependencies:
- cudatoolkit=11.1
- python=3.8
- pytorch
When I create an 'empty' python 3.8 environment and install the Conda packages from the command line instead of from an environment file, everything works fine:
$ conda env create --name bar python=3.8
...
$ conda activate bar
$ conda install pytorch cudatoolkit=11.1 -c pytorch -c nvidia
...
$ python
Python 3.8.10 | packaged by conda-forge | (default, May 11 2021, 07:01:05)
[GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import torch
>>> device = torch.device("cuda:0")
>>> t = torch.tensor(device=device, data=[0,1,2,3])
>>>
Can anyone tell what's going on here? It seems that Conda doesn't see the cudatoolkit=11.1 dependency while installing PyTorch from the environment file, but I have no idea how to fix it.
Just a few minutes after posting this question, I was able to figure out the solution. It turns out that it has to do with prioritizing Conda channels. The solution (which isn't well-documented by Anaconda) is to specify the correct channel for cudatoolkit and pytorch in environment.yml:
name: foo
channels:
- conda-forge
- nvidia
- pytorch
dependencies:
- nvidia::cudatoolkit=11.1
- python=3.8
- pytorch::pytorch
For conda version: 4.10.3, 4.11.0
conda --version to get your version
conda update -n base -c defaults conda to update your conda
This one worked for me:
in enviroment.yaml
name: nlp
channels:
- pytorch
dependencies:
- python=3.9
- numpy=1.21.5
- pandas=1.3.5
- spacy=3.2.1
- tensorflow=2.6.0
- pytorch=1.10.1
- cudatoolkit=11.3
in terminal
conda env create --file environment.yaml
conda activate nlp # use your env name from enviroment.yaml
python main.py
in main.py
import numpy as np
import pandas as pd
import spacy
import tensorflow as tf
import torch
print(f'np: {np.__version__}')
print(f'pd: {pd.__version__}')
print(f'spacy: {spacy.__version__}')
print(f'tf: {tf.__version__}')
print(f'torch: {torch.__version__}')
print(f'cuda enable: {torch.cuda.is_available()}')
print(f'current_device: {torch.cuda.current_device()}')
print(f'device: {torch.cuda.device(0)}')
print(f'device_count: {torch.cuda.device_count()}')
print(f'get_device_name: {torch.cuda.get_device_name(0)}')
output
np: 1.21.5
pd: 1.3.5
spacy: 3.2.1
tf: 2.6.0
torch: 1.10.1
cuda enable: True
current_device: 0
device: <torch.cuda.device object at 0x0000015156785EB0>
device_count: 1
get_device_name: NVIDIA GeForce GTX 1650 Ti
Yet another Import Error issue, I know...
I have been through many posts on stackoverflow about this issue but nothing seem to help
Here is the thing, I am not a windows person, I have used anaconda on mac and Linux for quite a while but now I have to train someone on Python with windows10.
So I tried to install Anaconda, thinking oh well that should be easy...
What I did :
Install miniconda3
Create a conda environment named 'first' - activated it
Install numpy scipy and geopandas (conda install -c conda-forge geopandas)
Start IPython
Typed import geopandas and got an Import Error ModuleNotFoundError: No module named 'geopandas'
Note that Geopandas and all packages appear in the pip list of my environment
So then I Import sys and typed sys.path with this result (sys.executable was fine)
['',
'C:\\Users\\mine\\Miniconda3\\Scripts',
'C:\\Users\\mine\\Miniconda3\\python36.zip',
'C:\\Users\\mine\\Miniconda3\\DLLs',
'C:\\Users\\mine\\Miniconda3\\lib',
'C:\\Users\\mine\\Miniconda3',
'C:\\Users\\mine\\Miniconda3\\lib\\site-packages',
'C:\\Users\\mine\\Miniconda3\\lib\\site-packages\\win32',
'C:\\Users\\mine\\Miniconda3\\lib\\site-packages\\win32\\lib',
'C:\\Users\\mine\\Miniconda3\\lib\\site-packages\\Pythonwin',
'C:\\Users\\mine\\Miniconda3\\lib\\site-packages\\IPython\\extensions',
'C:\\Users\\mine\\.ipython']
I believe I should see something about my conda environment in there
c:\users\mine\miniconda3\envs\first\
What did I do wrong? How do I fix it and why do you think that happened? Should I report it to anaconda as it seems to be a pretty annoying thing.
In case that helps :
(first) C:\Users\mine>conda info -a
active environment : first
active env location : C:\Users\mine\Miniconda3\envs\first
shell level : 2
user config file : C:\Users\mine\.condarc
populated config files :
conda version : 4.4.10
conda-build version : not installed
python version : 3.6.3.final.0
base environment : C:\Users\mine\Miniconda3 (writable)
channel URLs : https://repo.continuum.io/pkgs/main/win-64
https://repo.continuum.io/pkgs/main/noarch
https://repo.continuum.io/pkgs/free/win-64
https://repo.continuum.io/pkgs/free/noarch
https://repo.continuum.io/pkgs/r/win-64
https://repo.continuum.io/pkgs/r/noarch
https://repo.continuum.io/pkgs/pro/win-64
https://repo.continuum.io/pkgs/pro/noarch
https://repo.continuum.io/pkgs/msys2/win-64
https://repo.continuum.io/pkgs/msys2/noarch
package cache : C:\Users\mine\Miniconda3\pkgs
C:\Users\mine\AppData\Local\conda\conda\pkgs
envs directories : C:\Users\mine\Miniconda3\envs
C:\Users\mine\AppData\Local\conda\conda\envs
C:\Users\mine\.conda\envs
platform : win-64
user-agent : conda/4.4.10 requests/2.18.4 CPython/3.6.3 Windows/10 Windows/10.0.16299
administrator : False
netrc file : None
offline mode : False
# conda environments:
#
base C:\Users\mine\Miniconda3
first * C:\Users\mine\Miniconda3\envs\first
sys.version: 3.6.3 |Anaconda, Inc.| (default, Nov 8 ...
sys.prefix: C:\Users\mine\Miniconda3
sys.executable: C:\Users\mine\Miniconda3\python.exe
conda location: C:\Users\mine\Miniconda3\lib\site-packages\conda
conda-build: None
conda-env: C:\Users\mine\Miniconda3\Scripts\conda-env.exe
user site dirs:
CIO_TEST: <not set>
CONDA_DEFAULT_ENV: first
CONDA_PREFIX: C:\Users\mine\Miniconda3\envs\first
CONDA_PREFIX_1: C:\Users\mine\Miniconda3
CONDA_PROMPT_MODIFIER: (first)
CONDA_PYTHON_EXE: C:\Users\mine\Miniconda3\python.exe
CONDA_ROOT: C:\Users\mine\Miniconda3
CONDA_SHLVL: 2
HOMEPATH: \Users\mine
PATH: C:\Users\mine\Miniconda3\envs\first;C:\Users\mine\Miniconda3\envs\first\Library\mingw-w64\bin;C:\Users\mine\Miniconda3\envs\first\Library\usr\bin;C:\Users\mine\Miniconda3\envs\first\Library\bin;C:\Users\mine\Miniconda3\envs\first\Scripts;C:\Users\mine\Miniconda3\envs\first\bin;C:\Users\mine\Miniconda3;C:\Users\mine\Miniconda3\Library\mingw-w64\bin;C:\Users\mine\Miniconda3\Library\usr\bin;C:\Users\mine\Miniconda3\Library\bin;C:\Users\mine\Miniconda3\Scripts;C:\Users\mine\Miniconda3\bin;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\WINDOWS\System32\WindowsPowerShell\v1.0\;C:\Users\mine\AppData\Local\Microsoft\WindowsApps;
PSMODULEPATH: C:\Program Files\WindowsPowerShell\Modules;C:\WINDOWS\system32\WindowsPowerShell\v1.0\Modules
PYTHONIOENCODING: 1252
REQUESTS_CA_BUNDLE: <not set>
SSL_CERT_FILE: <not set>
You also need to install IPython into your new environment:
conda install ipython
otherwise it uses the ipython form "outside" the environment because it is a command line app and not necessary connected to an environment.
You can check IPython you use with:
where ipython
I have tried to install 'gtk' for python on Ubuntu (14.04). It seems to be installed OK. However I'm unable to import it.
$ python --version
Python 2.7.13 :: Anaconda 2.5.0 (64-bit)
$ python -c 'import gtk'
Traceback (most recent call last):
File "<string>", line 1, in <module>
ImportError: No module named gtk
$ which python
It seems to be installed OK...
$ conda list | grep -i gtk
conda list |grep -i gtk
gtk 3.16.6 2 ostrokach
I've only got one conda environment. PYTHONPATH and PYTHONHOME are unset.
$ conda info -a
Current conda install:
platform : linux-64
conda version : 4.3.24
conda is private : False
conda-env version : 4.3.24
conda-build version : 1.19.0
python version : 2.7.13.final.0
requests version : 2.18.3
root environment : /home/manohar/anaconda2 (writable)
default environment : /home/manohar/anaconda2
envs directories : /home/manohar/anaconda2/envs
/home/manohar/.conda/envs
package cache : /home/manohar/anaconda2/pkgs
/home/manohar/.conda/pkgs
channel URLs : https://conda.anaconda.org/brian-team/linux-64
https://conda.anaconda.org/brian-team/noarch
https://repo.continuum.io/pkgs/free/linux-64
https://repo.continuum.io/pkgs/free/noarch
https://repo.continuum.io/pkgs/r/linux-64
https://repo.continuum.io/pkgs/r/noarch
https://repo.continuum.io/pkgs/pro/linux-64
https://repo.continuum.io/pkgs/pro/noarch
config file : /home/manohar/.condarc
netrc file : None
offline mode : False
user-agent : conda/4.3.24 requests/2.18.3 CPython/2.7.13 Linux/3.13.0-123-generic debian/jessie/sid glibc/2.19
UID:GID : 1000:1000
# conda environments:
#
root * /home/manohar/anaconda2
sys.version: 2.7.13 |Anaconda 2.5.0 (64-bit)| (defaul...
sys.prefix: /home/manohar/anaconda2
sys.executable: /home/manohar/anaconda2/bin/python
conda location: /home/manohar/anaconda2/lib/python2.7/site-packages/conda
conda-build: /home/manohar/anaconda2/bin/conda-build
conda-convert: /home/manohar/anaconda2/bin/conda-convert
conda-develop: /home/manohar/anaconda2/bin/conda-develop
conda-env: /home/manohar/anaconda2/bin/conda-env
conda-index: /home/manohar/anaconda2/bin/conda-index
conda-inspect: /home/manohar/anaconda2/bin/conda-inspect
conda-metapackage: /home/manohar/anaconda2/bin/conda-metapackage
conda-pipbuild: /home/manohar/anaconda2/bin/conda-pipbuild
conda-server: /home/manohar/anaconda2/bin/conda-server
conda-sign: /home/manohar/anaconda2/bin/conda-sign
conda-skeleton: /home/manohar/anaconda2/bin/conda-skeleton
user site dirs:
CIO_TEST: <not set>
CONDA_DEFAULT_ENV: <not set>
CONDA_ENVS_PATH: <not set>
LD_LIBRARY_PATH: <not set>
PATH: /home/manohar/anaconda2/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/usr/bin:/usr/lib/jvm/default-java/bin
PYTHONHOME: <not set>
PYTHONPATH: <not set>
License directories:
/home/manohar/.continuum
/home/manohar/anaconda2/licenses
License files (license*.txt):
Package/feature end dates:
Any ideas as to what to check next? There are a plethora of gtk-related files inside ~/anancoda2:
~/anaconda2/share/gtk-3.0 (containing gtkbuilder.rng)
~/anaconda2/include/gtk-3.0 (many header files and subfolders)
~/anaconda2/bin/
gtk3-demo-application
gtk3-demo
gtk3-icon-browser
gtk3-widget-factory
gtk-encode-symbolic-svg
gtk-launch
gtk-query-immodules-3.0
gtk-update-icon-cache
~/anaconda2/lib/
gtk-2.0
gtk-3.0
libgtk-3.la
libgtk-3.so
libgtk-3.so.0
libgtk-3.so.1600.6
I'm trying to use Keras with the Theano backend inside a virtual environment. The error I'm getting is:
ERROR (theano.sandbox.cuda): Failed to compile cuda_ndarray.cu: libcublas.so.7.5: cannot open shared object file: No such file or directory
shown here:
(hyperproj) $ python
>> import theano
ERROR (theano.sandbox.cuda): Failed to compile cuda_ndarray.cu: libcublas.so.7.5: cannot open shared object file: No such file or directory
WARNING (theano.sandbox.cuda): The cuda backend is deprecated and will be removed in the next release (v0.10). Please switch to the gpuarray backend. You can get more information about how to switch at this URL:
https://github.com/Theano/Theano/wiki/Converting-to-the-new-gpu-back-end%28gpuarray%29
WARNING (theano.sandbox.cuda): CUDA is installed, but device gpu0 is not available (error: cuda unavailable)
This is what I've done so far:
$ screen
$ conda create -n hyperproj python=2.7 anaconda
$ source activate hyperproj
(hyperproj) $ conda install numpy
(hyperproj) $ conda install keras
(hyperproj) $ conda install theano
(hyperproj) $ cd $HOME
(hyperproj) $ cd .keras/
(hyperproj) $ vim keras.json
This is what keras.json looks like now (I changed the tensorflow backend to the theano backend):
{
"image_dim_ordering": "th",
"epsilon": 1e-07,
"floatx": "float32",
"backend": "theano"
}
I need to modify (or create) my .theanorc file:
(hyperproj) $ vim ~/.theanorc
After several tutorials, it looks like this:
[global]
floatX=float32
device=gpu0
[nvcc]
fastmath=True
[cuda]
root = /usr/local/cuda-7.5
Someone said I may need to check my .bashrc file and modify my $PATH. Here's what that looks like now (after, for example, this):
# Added by Canopy installer on 2016-06-20
# VIRTUAL_ENV_DISABLE_PROMPT can be set to '' to make bashprompt show that Canopy is active, otherwise 1
VIRTUAL_ENV_DISABLE_PROMPT=1 source /mnt/data/user/pkgs/enthought/canopy-1.5.1/bin/activate
# added by Anaconda2 2.5.0 installer
export PATH="/export/mlrg/user/anaconda2/bin:$PATH"
# added by Anaconda2 4.1.1 installer
export PATH="/mnt/data/user/pkgs/anaconda2/bin:$PATH"
#added to fix the theano libcudnn issue
export PATH=/usr/local/cuda/bin:/usr/local/cuda/lib64:$PATH
export LD_LIBRARY_PATH="/usr/local/cuda/lib64/:$LD_LIBRARY_PATH
I have CUDA 7.5, Ubuntu 14.04, g++ version 4.8.4.
How can I resolve the error?
Strangely, the problem was solved by logging out and logging back in again. When I logged in, I did this:
$ screen
$ source activate hyperproj
(hyperproj) $ which nvcc #gave me /usr/local/cuda/bin/nvcc
(hyperproj) $ cd /usr/local/cuda/lib64 #checked that I have libcublas.so
(hyperproj) $ cd $HOME
(hyperproj) $ python
Python 2.7.13 |Anaconda custom (64-bit)| (default, Dec 20 2016, 23:09:15)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-1)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
Anaconda is brought to you by Continuum Analytics.
Please check out: http://continuum.io/thanks and https://anaconda.org
>>> import keras
Using Theano backend.
WARNING (theano.sandbox.cuda): The cuda backend is deprecated and will be removed in the next release (v0.10). Please switch to the gpuarray backend. You can get more information about how to switch at this URL:
https://github.com/Theano/Theano/wiki/Converting-to-the-new-gpu-back-end%28gpuarray%29
Using gpu device 0: GeForce GTX TITAN Black (CNMeM is disabled, cuDNN not available)
I am on OSX el Capitan and doing Data Science.
For this I am using anaconda with Python 2.7
I used various envs successfully and was very happy in general with anaconda.
Now I wanted to do a new env (called tf for tensorflow) and install opencv 3.1 which I succeeded after several trials. So, if I open python, it prompts with
Python 2.7.13 |Continuum Analytics, Inc.| (default, Dec 20 2016, 23:05:08)
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
Anaconda is brought to you by Continuum Analytics.
Please check out: http://continuum.io/thanks and https://anaconda.org
and then I do
import cv2
print(cv2.__version__)
and it prompts me with 3.1.0
So far so good.
All this I do within my environment tf
But now I call a notebook by
jupyter notebook
and open a new notebook, import cv2, it does not like this
ImportError: No module named cv2
I cannot understand this and need help!
When I
conda list
I get all packages (see below partial paste)
jsonschema 2.5.1 py27_0
jupyter 1.0.0 py27_3
jupyter_client 5.0.0 py27_0
jupyter_console 5.1.0 py27_0
jupyter_core 4.3.0 py27_0
libpng 1.6.28 0 conda-forge
libtiff 4.0.6 7 conda-forge
markupsafe 0.23 py27_2
mistune 0.7.4 py27_0
mkl 2017.0.1 0
nbconvert 5.1.1 py27_0
nbformat 4.3.0 py27_0
notebook 4.4.1 py27_0
numpy 1.12.0 py27_0
opencv 3.1.0 np112py27_1 conda-forge
opencv3 3.1.0 py27_0 menpo
openssl 1.0.2k
I also add for info the output of the system when I do
conda info -a
I get
Current conda install:
platform : osx-64
conda version : 4.3.14
conda is private : False
conda-env version : 4.3.14
conda-build version : not installed
python version : 2.7.13.final.0
requests version : 2.12.4
root environment : /Users/peterhirt/anaconda (writable)
default environment : /Users/peterhirt/anaconda/envs/tf
envs directories : /Users/peterhirt/anaconda/envs
/Users/peterhirt/.conda/envs
package cache : /Users/peterhirt/anaconda/pkgs
/Users/peterhirt/.conda/pkgs
channel URLs : https://repo.continuum.io/pkgs/free/osx-64
https://repo.continuum.io/pkgs/free/noarch
https://repo.continuum.io/pkgs/r/osx-64
https://repo.continuum.io/pkgs/r/noarch
https://repo.continuum.io/pkgs/pro/osx-64
https://repo.continuum.io/pkgs/pro/noarch
config file : None
offline mode : False
user-agent : conda/4.3.14 requests/2.12.4 CPython/2.7.13 Darwin/15.6.0 OSX/10.11.6
UID:GID : 501:20
# conda environments:
#
tf * /Users/peterhirt/anaconda/envs/tf
root /Users/peterhirt/anaconda
sys.version: 2.7.13 |Anaconda 4.3.1 (x86_64)| (defaul...
sys.prefix: /Users/peterhirt/anaconda
sys.executable: /Users/peterhirt/anaconda/bin/python
conda location: /Users/peterhirt/anaconda/lib/python2.7/site-packages/conda
conda-build: None
conda-env: /Users/peterhirt/anaconda/bin/conda-env
conda-server: /Users/peterhirt/anaconda/bin/conda-server
user site dirs: ~/.local/lib/python2.7
CIO_TEST: <not set>
CONDA_DEFAULT_ENV: tf
CONDA_ENVS_PATH: <not set>
DYLD_LIBRARY_PATH: <not set>
PATH: /Users/peterhirt/anaconda/envs/tf/bin:/Users/peterhirt/anaconda/bin:/usr/local/bin:/Users/peterhirt/.npm-packages/bin:/Users/peterhirt/anaconda2/bin:/Users/peterhirt/google-cloud-sdk/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin
PYTHONHOME: <not set>
PYTHONPATH: <not set>
License directories:
/Users/peterhirt/.continuum
/Users/peterhirt/Library/Application Support/Anaconda
/Users/peterhirt/anaconda/licenses
License files (license*.txt):
Package/feature end dates:
Using Docker images helps the best in such cases since it encapsulates the environment. You can install Docker from here.
After pulling the image, you can use code like this in the shell:
docker run --rm -it -p 8888:8888 -v d:/Kaggles:/d datmo/kaggle:cpu
Run jupyter notebook inside the container
jupyter notebook --ip=0.0.0.0 --no-browser
This mounts the local directory onto the container having access to it.
Then, go to the browser and hit https://localhost:8888, and when I open a new kernel it's with Python 3.5.
You can find more information from here.
You can also try using datmo in order to easily setup environment and track machine learning projects to make experiments reproducible. You can run datmo task command as follows for setting up jupyter notebook,
datmo task run 'jupyter notebook' --port 8888
It sets up your project and files inside the environment to keep track of your progress.