I'm struggling to just install and import caffe on my windows 10's Anaconda (conda 4.7.11). I really don't know why such a well-known deep-learning package is that hard to just install!
First, I tried to install it. With huge difficulties, I did the following and at last, apparently installation was successful:
conda create -n caffe python=3.7
// activate the environment REM Depending on your conda version you may have to use activate caffe
conda activate caffe
// add the anaconda channel to resolve all dependencies
conda config --add channels anaconda
// Install caffe with cuda support or install caffe-cpu if you do not want CUDA support
conda install caffe-cpu -c willyd
And now, just trying to import caffe, I get this error:
>>> import caffe
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Users\user\AppData\Local\Continuum\anaconda3\envs\caffe\lib\site-packages\caffe\__init__.py", line 1, in <module>
from .pycaffe import Net, SGDSolver, NesterovSolver, AdaGradSolver, RMSPropSolver, AdaDeltaSolver, AdamSolver, NCCL, Timer
File "C:\Users\user\AppData\Local\Continuum\anaconda3\envs\caffe\lib\site-packages\caffe\pycaffe.py", line 13, in <module>
from ._caffe import Net, SGDSolver, NesterovSolver, AdaGradSolver, \
ImportError: DLL load failed: The specified module could not be found.
What should I do? How to fix it?
BTW, is Caffe2 different from Caffe?
Caffe2 is built on caffe but yes it is different. This link : http://gianni.rosagallina.com/en/posts/2018/10/04/caffe2-gpu-windows-1.html#step-4-build provides a good overview of what steps need to be taken to install caffe2. Incase you hit and error you cannot resolve, you can report the problems on the pytorch git and the team will help you out.
Related
I followed this tutorial, https://www.youtube.com/watch?v=ykCY_tJbhNw to install tensorflow in a mac (M1 processor). However when I import tensrflow I get the following error. Any ideas what it might be?
The steps include,
installing brew (was alreadyy installed)
install conda (was already installed)
installing miniforge
download tensorflow-macos from github page
conda activate mlEnv
/Users/imantha/Software/tensorflow_macos/install_venv.sh -p
When prompted to input the path : /Users/imantha/Software/anaconda3/envs/mlEnv/
and select yes to install tensorflow
tensorflow was successfully installed and when typed conda list, the package is listed.
However when I start python session in the shell and type import tensorflow as tf, I get the following error
Traceback (most recent call last):
File "/Users/imantha/Software/anaconda3/envs/mlEnv/lib/python3.8/site-packages/tensorflow/python/pywrap_tensorflow.py", line 64, in <module>
from tensorflow.python._pywrap_tensorflow_internal import *
ImportError: dlopen(/Users/imantha/Software/anaconda3/envs/mlEnv/lib/python3.8/site-packages/tensorflow/python/_pywrap_tensorflow_internal.so, 6): no suitable image found. Did find:
/Users/imantha/Software/anaconda3/envs/mlEnv/lib/python3.8/site-packages/tensorflow/python/_pywrap_tensorflow_internal.so: mach-o, but wrong architecture
/Users/imantha/Software/anaconda3/envs/mlEnv/lib/python3.8/site-packages/tensorflow/python/_pywrap_tensorflow_internal.so: mach-o, but wrong architecture
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Users/imantha/Software/anaconda3/envs/mlEnv/lib/python3.8/site-
packages/tensorflow/__init__.py", line 41, in <module>
from tensorflow.python.tools import module_util as _module_util
File "/Users/imantha/Software/anaconda3/envs/mlEnv/lib/python3.8/site-
packages/tensorflow/python/__init__.py", line 39, in <module>
from tensorflow.python import pywrap_tensorflow as _pywrap_tensorflow
File "/Users/imantha/Software/anaconda3/envs/mlEnv/lib/python3.8/site-
packages/tensorflow/python/pywrap_tensorflow.py", line 83, in <module>
raise ImportError(msg)
ImportError: Traceback (most recent call last):
File "/Users/imantha/Software/anaconda3/envs/mlEnv/lib/python3.8/site-
packages/tensorflow/python/pywrap_tensorflow.py", line 64, in <module>
from tensorflow.python._pywrap_tensorflow_internal import *
ImportError:
dlopen(/Users/imantha/Software/anaconda3/envs/mlEnv/lib/python3.8/site-
packages/tensorflow/python/_pywrap_tensorflow_internal.so, 6): no suitable image
found. Did find:
/Users/imantha/Software/anaconda3/envs/mlEnv/lib/python3.8/site-
packages/tensorflow/python/_pywrap_tensorflow_internal.so: mach-o, but wrong
architecture
/Users/imantha/Software/anaconda3/envs/mlEnv/lib/python3.8/site-
packages/tensorflow/python/_pywrap_tensorflow_internal.so: mach-o, but wrong
architecture
Failed to load the native TensorFlow runtime.
See https://www.tensorflow.org/install/errors
for some common reasons and solutions. Include the entire stack trace
above this error message when asking for help.
There are quite a few things to do in order to install TensorFlow on the Macbook M1 chip:
First, install Python 3.8.9 (I have tried on this version, not sure about any other versions)
Go to your project dir
install hdf5 by running brew install hdf5 if you do not have brew, you can download it here: https://brew.sh/
Above hdf5 install will spit out its location: use it and run:
export HDF5_DIR=<hdf5_location> for version 1.13.0 (my case) the following:
export HDF5_DIR=/opt/homebrew/Cellar/hdf5/1.13.0
create a virtual environment using virtualenv venv (choose whatever method for creating virtual environment or do not create virtual environment, it's fine. To install virtualenv, run pip3 install virtualenv)
(Skip this if not using virtual environment) Activate virtual environment by running: source venv/bin/activate
Now, we have everything require to install Tensorflow on mac with M1 chip including GPU version
To install Tensorflow and related libs, run following:
(If using virtual env, run following)
venv/bin/pip install --no-binary=h5py h5py
venv/bin/pip install tensorflow-macos
venv/bin/pip install tensorflow-metal
Not using virtual env, run following:
pip3 install --no-binary=h5py h5py
pip3 install tensorflow-macos
pip3 install tensorflow-metal
Now, your TensorFlow installation has been completed!
Try it out:
import tensorflow as tf
print("TF version:", tf.__version__)
print("Number GPUs Available: ", len(tf.config.experimental.list_physical_devices('GPU')))
print("Number CPUs Available: ", len(tf.config.experimental.list_physical_devices('CPU')))
Build Tensorflow from source by following the instructions mentioned here.
See this comment.
OR,
Try the workaround mentioned here.
The below way is the simplest and efficient way from which we can install tensorflow in Mac M1
NOTE: Mac M1 has ARM64 arch not X86_64 so we need to be very careful while downloading packages. Anaconda is available for Mac M1 users : https://repo.anaconda.com/archive/Anaconda3-2022.05-MacOSX-arm64.sh
Following are the steps to victory:
Download Anaconda from above link
create virtual env ( Optional )
pip install tensorflow-macos
Thats it!
Reference video link which may be useful : https://www.youtube.com/watch?v=5DgWvU0p2bk
If you want to use an Anaconda distribution, you might want to consider Miniconda3. Currently, I'm not sure whether it's possible to install TensorFlow using Anaconda.
Download Miniconda3 using this link. Both the .bash and the .pkg files should do - choose one of them. However, for M1, only the .bash file seem to have worked, while for M2 both worked.
If you used the .bash file above, then do the following to start the installation:
chmod +x ~/Downloads/Miniforge3-MacOSX-arm64.sh
sh ~/Downloads/Miniforge3-MacOSX-arm64.sh
Create a conda environment with Python 3.9: conda create -n tf_env python=3.9 (latest Python version didn't work as I'm writing this).
Install TensorFlow and dependencies:
conda install -c apple tensorflow-deps
python -m pip install tensorflow-macos
python -m pip install tensorflow-metal
At this point, you should make sure that the Numpy version is up to date to not run into errors when importing TensorFlow. I used version 1.23.3. python -m pip install numpy --upgrade.
Install jupyter notebook using conda install notebook.
If everything has been installed without errors, running the following code in a jupyter notebook should correctly import TensorFlow and print out the version as well as verifying the presence of a GPU.
import tensorflow as tf
gpu = len(tf.config.list_physical_devices('GPU')) > 0
print(f"TensorFlow Version: {tf.__version__}")
print("GPU is", "available" if gpu else "NOT available")
Discription
I am trying to install pytorch 0.4 on Windows10.
My enviroment settings:
- Windows10
- cuda9.0
- python 3.6
- pytorch 0.4
- anaconda
I tried by using both conda install -n myenv and pip install $path:whl and both failed.
Error
>>> import torch
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "D:\Anaconda\envs\py3.6_pytorch0.4\lib\site-packages\torch\__init__.py", line 80, in <module>
from torch._C import *
ImportError: DLL load failed:
I found the related a issue #4518 under pytorch but answers under that issue do not work for me.
What I have tried
add all conda related path to environment path
change the directory (cd)
install vs_runtime under this conda env
None of those works.
But if I install pytorch under conda's base environment, it works well.
So what's going on here?
Update:
When we tried to install pytorch on windows, a lot of packages need to be installed at the same time. We can simply follow the steps on official website [link](https://pytorch.org/），for python 3.6 and cuda9.0 the installation command is as
conda install pytorch torchvision cudatoolkit=9.0 -c pytorch
If we want to install former version of pytorch, we can assign the version, eg. 0.4 as
conda install pytorch=0.4 torchvision cudatoolkit=9.0 -c pytorch
After you create the new environment with conda, execute conda install -c pytorch pytorch to install pytorch.
pip does not work that well with external, non-Python dependencies. Not unlikely in your case path to the DLL is not set correctly (just a guess).
I've used the following tutorial to install Python, numpy, scipy and matplotlib:
https://penandpants.com/2012/03/01/install-python-2/
I downloaded each and installed by double-clicking. I'm using a Mac with OS X Yosemite Version 10.10.5
When I try to import numpy, scipy and matplotlib in IDLE I get the follow error message:
`>>> import numpy
Traceback (most recent call last):
File "<pyshell#2>", line 1, in <module>
import numpy
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/__init__.py", line 137, in <module>
import add_newdocs
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/add_newdocs.py", line 9, in <module>
from numpy.lib import add_newdoc
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/lib/__init__.py", line 4, in <module>
from type_check import *
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/lib/type_check.py", line 8, in <module>
import numpy.core.numeric as _nx
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/core/__init__.py", line 5, in <module>
import multiarray
ImportError: dlopen(/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/core/multiarray.so, 2): no suitable image found. Did find:
/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/core/multiarray.so: no matching architecture in universal wrapper`
Can anyone please explain to me how to resolve this?
Thanks
use terminal and type
sudo pip install numpy
sudo pip install matplotlib
sudo pip install scipy
for python3+ use pip3
I can't say for sure what's wrong with your installation, but I suspect it's due to a some PATH-conusion... Python 2.7 and Numpy should be installed on your Mac by default.
If you run the command which python, you'll see which executable the python-commands invokes... is it the same as the one you installed?
If you write python in the terminal, and press tab twice, do you see multiple options of installed python versions?
Alternative method of installation
I prefer the Anaconda Python Distribution, which you can download like the full version or minimal version, which is the one I prefer.
If you install this, you should consider uninstalling the one you just installed.
Here's how to install the mini version:
Download the installer, and run it by opening a terminal in the folder with the installer, and running the command bash Miniconda2-latest-MacOSX-x86_64.sh. This will start the text-based installer, and if you read the instructions on screen, you'll be fine.
Notice that you have to scroll to the end of the licence agreement, before you can write yes when asked to accept the terms stated.
Say yes when it asked you if you'd like it to add Anaconda Python to your path.
Anaconda Python is different from other python distributions in the regard that it comes with it's own package and environment manager conda. With conda you can have multiple versions of Python installed at the same time (or several environments of the same Python version, but with different packages or versions of packages).
To create a new environment with the name py27, which is using Python 2.7, write conda create -n py27 python=2.7 --yes. Once created, activate the environment by source activate py27.
With the new environment now active, install the packages you need by using conda: conda install scipy matplotlib --yes.
I'd also recommend installing Ipython, which is an improved shell, and possibly the Jupyter notebook: conda install ipython --yes and conda install notebook --yes.
Hope it helps.
I have created a new conda env and attempted to install numpy into it. After activating the env, I used conda install numpy. However, when I start python and attempt to import numpy I get the following import error:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/tjim/anaconda3/envs/tensorflow/lib/python3.6/site-packages/numpy/__init__.py", line 112, in <module>
import ctypes
File "/home/tjim/anaconda3/envs/tensorflow/lib/python3.6/ctypes/__init__.py", line 7, in <module>
from _ctypes import Union, Structure, Array
ImportError: /home/tjim/anaconda3/envs/tensorflow/lib/python3.6/lib-dynload/_ctypes.cpython-36m-x86_64-linux-gnu.so: undefined symbol: PySlice_Unpack
A similar error appears when I try to install scipy. I have also tried uninstalling them both and letting conda install numpy as a dependency of pandas but that didn't work either. What am I doing wrong? What info should I provide to help fix the problem? Thanks!
I deleted my conda virtual env and started a fresh one, cloning my original anaconda install (which had a working numpy) using conda create -n yourenvname python=3.6 anaconda as a workaround.
I'm trying to create an LMDB data base file in Python to be used with Caffe according to this tutorial. The commands import numpy as np and import caffe run perfectly fine. However, when I try to run import lmdb and import deepdish as dd, I'm getting the following errors:
>>> import lmdb
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named lmdb
>>> import deepdish as dd
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named deepdish
I'm running Python 2.7.9 through Anaconda 2.2.0 (64-bit) on Ubuntu 14.04. While installing the dependencies for Caffe according to this page, I've already installed the lmdb package through sudo apt-get install liblmdb-dev.
Any ideas why this error might be occuring?
Well, the apt-get install liblmdb-dev might work with bash (in the terminal) but apparently it doesn't work with Anaconda Python. I figured Anaconda Python might require it's own module for lmdb and I followed this link. The Python installation for lmdb module can be performed by running the command pip install lmdb in the terminal. And then import lmdb in Python works like a charm!
The above installation commands may require sudo.
If you're using Anaconda, then this can solve your problem (it worked for me):
conda install -c https://conda.binstar.org/dougal lmdb
For Anaconda users, installing python-lmdb package from conda-forge should fix the lmdb import error:
conda install -c conda-forge python-lmdb
This was tested on conda 4.5.11 on an lxc-containerized system running Ubuntu 18.04.
Note that there is a conda package named lmdb (without python-), installable via:
conda install -c conda-forge lmdb
that does not fix the import error.