Installing Open3d-Ml with Pytorch (on MacOs) - python

I created a virtualenv with python 3.10 and installed open3d and PyTorch according to the instructions on open3d-ml webpage: Open3d-ML but when I tested it with import open3d.ml.torch I get the error:
Exception: Open3D was not built with PyTorch support!
Steps to reproduce
python3.10 -m venv .venv
source .venv/bin/activate
pip install --upgrade pip
pip install open3d
pip install torch torchvision torchaudio
Error
% python -c "import open3d.ml.torch as ml3d"
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Users/xx/.venv/lib/python3.10/site-packages/open3d/ml/torch/__init__.py", line 34, in <module>
raise Exception('Open3D was not built with PyTorch support!')
Exception: Open3D was not built with PyTorch support!
Environment:
% python3 --version
Python 3.10.9
% pip freeze
open3d==0.16.1
torch==1.13.1
torchaudio==0.13.1
torchvision==0.14.1
OS
macOS 12.6
Kernel Version: Darwin 21.6.0
I also checked below similar issues but they don't have answers:
https://github.com/isl-org/Open3D/discussions/5849
https://github.com/isl-org/Open3D-ML/issues/557
Open3D-ML and pytorch
According to this issue 5849 the problem can't be related only to MacOs because, in a docker with Ubuntu20.04, there is a similar error.
Does anyone know how we can tackle this?

You will need to enable PyTorch / Tensorflow when building with cmake - have a look at https://medium.com/#kidargueta/installing-open3d-ml-for-3d-computer-vision-with-pytorch-d640a6862e19 and http://www.open3d.org/docs/release/compilation.html
cmake -DBUILD_CUDA_MODULE=ON -DGLIBCXX_USE_CXX11_ABI=OFF -DBUILD_PYTORCH_OPS=ON -DBUILD_CUDA_MODULE=ON -DBUNDLE_OPEN3D_ML=ON -DOPEN3D_ML_ROOT=Open3D-ML -DBUILD_JUPYTER_EXTENSION:BOOL=ON -DBUILD_WEBRTC=ON -DOPEN3D_ML_ROOT=https://github.com/isl-org/Open3D-ML.git -DPython3_ROOT=/path/to/your/conda-env/bin/python ..

Finally, I decided to build Open3D from the source for Mac M1. I followed almost the official open3d page and thanks to this medium in one of the replies.
Build Open3d-ml with Pytorch on Mac M1
For the OS environment see the main question.
conda create --name open3d-ml-build python==3.10
conda activate open3d-ml-build
# install pytorch from pytorch.org
conda install pytorch torchvision torchaudio -c pytorch
# now clone open3d in a desired location
git clone --branch v0.16.1 git#github.com:isl-org/Open3D.git ./foo/open3d-0.16-build
cd open3d-0.16-build
mkdir build && cd build
git clone git#github.com:isl-org/Open3D-ML.git
Now make sure you are in the activeted conda env.
Build
(takes very long and a lot of memory)
Note on Mac M1 you don't have Cuda but Metal Performance Shaders (MPS) so I made CUDA Flags OFF in the cmake configuration.
which python3
>> /Users/XX/miniconda3/envs/open3d-ml-build/bin/python3
# in the build direcotry
cmake -DBUILD_CUDA_MODULE=OFF -DGLIBCXX_USE_CXX11_ABI=OFF \
-DBUILD_PYTORCH_OPS=ON -DBUILD_CUDA_MODULE=OFF \
-DBUNDLE_OPEN3D_ML=ON -DOPEN3D_ML_ROOT=Open3D-ML \
-DBUILD_JUPYTER_EXTENSION:BOOL=OFF \
-DPython3_ROOT=/Users/XX/miniconda3/envs/open3d-ml-build/bin/python3 ..
make -j$(sysctl -n hw.physicalcpu) [verbose=1]
If it fails, try it again or run it with verbose and look for fatal error.
Install
# Install pip package in the current python environment
make install-pip-package
# if error: Module Not found yapf
pip install yapf
# Create Python package in build/lib
make python-package
# Create pip wheel in build/lib
# This creates a .whl file that you can install manually.
make pip-package
sanity check
Again in the activated conda environment
# if not installed
pip install tensorboard
python3 -c "import open3d; import open3d.ml.torch"
pip freeze | grep torch
torch==1.13.1
torchaudio==0.13.1
torchvision==0.14.1
If you don't get any errors you should be good to go.

Related

Error when importing torch_geometric in Python 3.9.7

I'm trying to install torch_geometric in a conda environment but I'm getting the following werror whenever I try to:
import torch_geometric
Error:
OSError: dlopen(/Users/psanchez/miniconda3/envs/playbook/lib/python3.9/site-packages/libpyg.so, 0x0006): Library not loaded: /usr/local/opt/python#3.10/Frameworks/Python.framework/Versions/3.10/Python
Referenced from: <95F9BBA5-21FB-3EA5-9028-172B745E6ABA> /Users/psanchez/miniconda3/envs/playbook/lib/python3.9/site-packages/libpyg.so
Reason: tried: '/usr/local/opt/python#3.10/Frameworks/Python.framework/Versions/3.10/Python' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/usr/local/opt/python#3.10/Frameworks/Python.framework/Versions/3.10/Python' (no such file), '/usr/local/opt/python#3.10/Frameworks/Python.framework/Versions/3.10/Python' (no such file), '/Library/Frameworks/Python.framework/Versions/3.10/Python' (no such file), '/System/Library/Frameworks/Python.framework/Versions/3.10/Python' (no such file, not in dyld cache)
This is how I installed the conda envrionment:
onda create --name playbook python=3.9.7 --no-default-packages
conda activate playbook
pip install torch==1.13.1 torchvision==0.14.1
pip install pyg-lib torch-scatter torch-sparse torch-cluster torch-spline-conv torch-geometric -f https://data.pyg.org/whl/torch-1.13.0+cpu.html
Any idea how to solve this error?
Thanks a lot in advance!
If you check your error message, on the line "Referenced from" you can see Python version 3.9 but on the line "Reason tried", you have Python version 3.10.
So i think you are using a Python environment 3.10 with Python 3.9 or the opposite. You should recreate your environment cleanly.
I had this problem as well--I don't know why it wants Python 3.10 specifically. But I was able to resolve it by installing a global version Python 3.10:
https://www.python.org/downloads/macos/
This allows it to access Python 3.10 outside your virtual environment.
I had this problem as well.I used python3.7 it worked:
conda create -n pygeometric python=3.7
conda activate pygeometric
conda install pytorch torchvision -c pytorch
python -c "import torch; print(torch.version)"
conda install -y clang_osx-64 clangxx_osx-64 gfortran_osx-64
MACOSX_DEPLOYMENT_TARGET=10.9 CC=clang CXX=clang++
pip install torch_scatter
python -c "import torch_scatter; print(torch_scatter.version)"
pip install torch_sparse
python -c "import torch_sparse; print(torch_sparse.version)"
pip install torch_cluster
python -c "import torch_cluster; print(torch_cluster.version_)"
pip install torch-spline-conv
python -c "import torch_spline_conv;print(torch_spline_conv.version)"
pip install torch_geometric
python -c "import torch_geometric;print(torch_geometric.version)"

OSError when doing "from keras.preprocessing.image import ImageDataGenerator" in M1 Mac [duplicate]

I am trying to install Tensorflow on my MacBook Pro with the M1 chip. The operating system of my MacBook is MacOS Big Sur Version 11.0.
In order to install Tensorflow to use it with Python, I have followed this tutorial, which says that I have to do the following:
Install Homebrew.
Download MiniForge3 for macOS arm64 chips (link provided in the webpage).
Install MiniForge3 using:
chmod +x ~/Downloads/Miniforge3-MacOSX-arm64.sh
sh ~/Downloads/Miniforge3-MacOSX-arm64.sh
source ~/miniforge3/bin/activate
Create a folder to set up an environment for Tensorflow.
mkdir tensorflow-test
cd tensorflow-test
Make and activate Conda environment.
conda create --prefix ./env python=3.9.7
conda activate ./env
Install Tensorflow dependencies.
conda install -c apple tensorflow-deps
python -m pip install tensorflow-macos
python -m pip install tensorflow-metal
After this, I open a Jupyter Notebook and I try to import tensorflow, but this error shows up:
OSError: dlopen(/Users/blancoarnau/tensorflow-test/env/lib/python3.9/site-packages/tensorflow/python/platform/../../core/platform/_cpu_feature_guard.so, 6): Symbol not found: __ZNKSt3__115basic_stringbufIcNS_11char_traitsIcEENS_9allocatorIcEEE3strEv
Referenced from: /Users/blancoarnau/tensorflow-test/env/lib/python3.9/site-packages/tensorflow/python/platform/../../core/platform/_cpu_feature_guard.so (which was built for Mac OS X 12.3)
Expected in: /usr/lib/libc++.1.dylib
As you can see in this screenshot:
Do you have an idea why this is happening?
check the message details:
(which was built for Mac OS X 12.3)
you need to upgrade macOS to 12.3

how do I install a specific version of the rdkit library for Python2?

I need to install a version of the rdkit library released prior to 2019, when support for Python 2 was removed. This is needed to work with this library: https://github.com/brain-research/deep-molecular-massspec
I have downloaded the library from the git page, eg. https://github.com/rdkit/rdkit/releases/tag/Release_2018_09_1, and tried using pip to install from that.
sudo pip install rdkit-Release_2018_09_1b1.tar.gz
I get the following error:
Processing ./rdkit-Release_2018_09_1b1.tar.gz
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "", line 1, in
IOError: [Errno 2] No such file or directory: '/tmp/pip-ohIcaj-build/setup.py'
---------------------------------------- Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-ohIcaj-build
I have tried installing the specific version using pip too:
sudo pip install rdkit==2018.09.01
Which gives:
Collecting rdkit==2018.09.01 Could not find a version that satisfies
the requirement rdkit==2018.09.01 (from versions: ) No matching
distribution found for rdkit==2018.09.01
Can someone tell me how to do this?
#paisanco is correct, attempting to install rdkit with pip will not work. The easiest way to install rdkit is by using Anaconda unless you want to build from source.
If you have Anaconda installed you can create a python 2.7 virtual environment:
conda create --name test-env python=2.7
You can then activate it:
conda activate test-env
And then install the rdkit version you require:
conda install -c rdkit rdkit=2018.09.1
Using Python:
import rdkit
print rdkit.__version__
[Out]: '2018.09.1'
Create a new conda environment with python 2.7.15:
conda create -n py27_rdkit python=2.7.15 ipython
Activate environment (python2.7)
conda activate py27_rdkit
Now in the py27_protac environment, install older version of rdkit that won't gripe about python2.7:
conda install -c conda-forge rdkit rdkit=2018.09.1
The conda install command in answer above: 'conda install -c rdkit rdkit=2018.09.1' failed due numerous conflicts.
The problem is what you downloaded, according to that site, is a tar archive containing the source code for that library, not a pip package.
So attempting to install it using pip will not work.
The RDKit project home page gives other options for installing 1) from within an Anaconda conda virtual environment 2) from the source code (what you downloaded) for Windows, Linux, and Mac.
Those instructions are at RDKit installation instructions
conda create -n my_env python=3.7
conda activate my_env
conda install numpy matplotlib
conda install cmake cairo pillow eigen pkg-config
conda install boost-cpp boost py-boost
and download rdkit package https://anaconda.org/rdkit/rdkit/files
# finally
conda install rdkit-2020.09.1b1.0-py37hd50e099_1.tar.bz2

Error of Installing dlib in anaconda

I'm trying to install Dlib in anaconda python with this line:
conda install -c menpo dlib
and I'm getting error like this:
UnsatisfiableError: The following specifications were found to be in conflict:
- dlib
- xlwt
Use "conda info <package>" to see the dependencies for each package.
How can I solve this problem?
Path of core-meta: C/users/{xxx}/anaconda3/core-meta
Delete core-meta then run the following commands:
Conda update --all
Conda install dlib
These commands require an internet connection so keep wifi on during the command running process.
Here xxx is the username on your computer.
If you do not need the xlwt-package for your project, just create a new conda environment and install the dlip package locally in that environment.
you can also try following:
brew install cmake
brew install boost
then create a virtual environment using conda command
conda create -n dlib python=3.6 anaconda
Activate the environment
source activate dlib
run the following command
conda install pip
and finally install dlib using below command
pip install dlib
This worked for me.
For anaconda or miniconda3, please see the following installation procedures. It will comply with Python 3.7 in this way.
1.Install cmake with git
$ sudo apt-get install git cmake
2. Enter into the the directory
Please change user to your own username
$ cd /home/user/dlib
3. Compile dlib
$ python setup.py install
If meeting with the issue that you could not put the import statements of setuptools behind the import statement of distutils, please change the placement oder in setup.py in the directory of dlib.
from setuptools import setup, Extension
from setuptools.command.build_ext import build_ext
from distutils import log
from distutils.version import LooseVersion
4. Run the dlib unit test suite in the current working directory
Please note there is a sub-directory of dlib inlcuded in the directory of dlib.
$ cd dlib/test
$ mkdir build
$ cd build
$ cmake ..
$ cmake --build . --config Release
$./dtest --runall
Cheers.
on mac system :
you have to do it via brew
brew install cmake
brew install boost
brew install boost-python --with-python3
After installing above packages if you need you can create your environment and proceed with dlib installation or you can simply run next step.
The dlib library doesn’t have any real Python prerequisites, but if you plan on using dlib for any type of computer vision or image processing, These are few libraries which you can install for dlib :
* NumPy
* SciPy
* scikit-image
pip install numpy
5 pip install Scipy
pip install scikit-image
pip install dlib
you can check for more details on this link:
https://www.pyimagesearch.com/2017/03/27/how-to-install-dlib/
You have to try by putting just:
conda install dlib

OpenCV not working properly with python on Linux with anaconda. Getting error that cv2.imshow() is not implemented

This is the exact error that I am getting. My OS is Ubuntu 16.10.
OpenCV Error: Unspecified error (The function is not implemented. Rebuild the library with Windows, GTK+ 2.x or Carbon support. If you are on Ubuntu or Debian, install libgtk2.0-dev and pkg-config, then re-run cmake or configure script) in cvShowImage, file /feedstock_root/build_artefacts/work/opencv-3.1.0/modules/highgui/src/window.cpp, line 545
Traceback (most recent call last):
File "untitled.py", line 7, in
cv2.imshow('image',img)
cv2.error: /feedstock_root/build_artefacts/work/opencv-3.1.0/modules/highgui/src/window.cpp:545: error: (-2) The function is not implemented. Rebuild the library with Windows, GTK+ 2.x or Carbon support. If you are on Ubuntu or Debian, install libgtk2.0-dev and pkg-config, then re-run cmake or configure script in function cvShowImage
my code is:
import numpy as np
import cv2
# Load an color image in grayscale
img = cv2.imread('0002.png',0)
cv2.imshow('image',img)
cv2.waitKey(0)
cv2.destroyAllWindows()
0002.png is an image in the same directory as the program.
I first installed anaconda with python 3.5, then I installed opencv by using the command
conda install -c conda-forge opencv
I installed libgtk2.0-dev just as the error said to but I still get the same error.
Any help would be much appreciated. I've been trying to solve this for several hours.
1.The easiest way:
conda remove opencv
conda update conda
conda install --channel menpo opencv
or (for OpenCV 3.1) :
conda install -c menpo opencv3
2.And if u don't want to do this, you can try to use matplotlib .
import cv2
import matplotlib.pyplot as plt
img = cv2.imread('img.jpg',0)
plt.imshow(img, cmap='gray')
plt.show()
3.Or try to build library by your own with option WITH_GTK=ON , or smth like that.
Update - 18th Jun 2019
I got this error on my Ubuntu(18.04.1 LTS) system for openCV 3.4.2, as the method call to cv2.imshow was failing. I am using anaconda. Just the below 2 steps helped me resolve:
conda remove opencv
conda install -c conda-forge opencv=4.1.0
If you are using pip, you can try
pip install opencv-contrib-python
I have had to deal with this issue a couple of times, this is what has worked consistently thus far:
conda remove opencv
conda install -c menpo opencv
pip install --upgrade pip
pip install opencv-contrib-python
πŸš€ Working method (πŸ‘¨β€πŸ”¬ tested on April 19, 2019)
🐞 These error happen because of conda.
🏹 Open Anaconda Prompt and remove conda opencv if installed
πŸ‘¨β€πŸ’» conda remove opencv
πŸ“’ If you have conda env, firstly activate it conda activate <your_env_name>
⏬ After install opencv via pip (click here to offical info)
πŸ‘¨β€πŸ’» pip install opencv-contrib-python
πŸ“’ if pip haven't installed, use conda install pip command.
If you installed OpenCV using the opencv-python pip package at any point in time, be aware of the following note, taken from https://pypi.python.org/pypi/opencv-python
IMPORTANT NOTE
MacOS and Linux wheels have currently some limitations:
video related functionality is not supported (not compiled with FFmpeg)
for example cv2.imshow() will not work (not compiled with GTK+ 2.x or Carbon support)
Also note that to install from another source, first you must remove the opencv-python package
For me (Arch Linux, Anaconda with Python 3.6), installing from the suggested channels menpo or loopbio did not change anything. My solution (see related question) was to
install pkg-config (sudo pacman -Syu pkg-config),
remove opencv from the environment (conda remove opencv) and
re-install opencv from the conda-forge channel (conda install -c conda-forge opencv)
conda list now returns opencv 3.3.0 py36_blas_openblas_203 [blas_openblas] conda-forgeand all windows launched using cv2 are working fine.
I followed this tutorial (OpenCV GTK+2.x error) and did the following. It worked for me :
install the packages : libgtk2.0-dev and pkg-config
cd to your opencv directory
mkdir Release
cd Release
Run the command : cmake -D CMAKE_BUILD_TYPE=RELEASE -D CMAKE_INSTALL_PREFIX=/usr/local -D WITH_TBB=ON -D BUILD_NEW_PYTHON_SUPPORT=ON -D WITH_V4L=ON -D INSTALL_C_EXAMPLES=ON -D INSTALL_PYTHON_EXAMPLES=ON -D BUILD_EXAMPLES=ON -D WITH_QT=ON -D WITH_GTK=ON -D WITH_OPENGL=ON ..
make
sudo make install
Notice that it is complaining for libgtk2.0-dev and pkg-config. Here is the solution. Uninstall your existing openCV installation.
conda remove opencv3
Install these packages before installing opencv-
conda install gtk2 pkg-config
Now install opencv from menpo
conda install -c https://conda.anaconda.org/menpo opencv3
If you are running inside docker then you may get this error.Solution uninstall current and install the headless one
pip install opencv-python==3.4.5.20
pip install opencv-contrib-python==3.4.5.20
If you are writing to the image and displaying it, you may need the following
apt-get update && apt-get install -y libglib2.0-0 libsm6 libxext6 libxrender1
And if you are wondering how to get the display from Docker, it is possible via X11 in your host
My ubuntu 18.04 machine is running at AWS.
What helped me was (link):
pip uninstall opencv-python
pip install opencv-python==4.1.2.30
Afterwards I got the error: " : cannot connect to X server "
Finally, I managed to make it work by installing MobaXTerm (reference here).
FYI:
I connect to AWS with WinSCP. If you connect to AWS via something like WinSCP, the MobaXTerm interface lets you connect to the "WinSCP temporary session" if you click at "Sessions", which is very convenient. The session screen basically replaces the console but allows to display the image via a pop-up window.
In case you also experience the error ASSERT: β€œfalse” in file qasciikey.cpp, line 501 once cv.imshow() or similar is executed, go to the MobaXTerm interface, click Settings -> Configuration -> X11, uncheck "Unix-compatible keyboard" (reference here).
I used pip to install opencv-python. (https://pypi.org/project/opencv-python/)
1) Remove the opencv package from conda:
>> conda remove opencv
2) To your env.yml file add this:
...
dependencies:
- numpy
- pytest
...
- pip:
- opencv-python
Remove opencv from anaconda=
conda remove opencv
Then, reinstall opencv using pip:
pip install opencv
This is working for me.
My Envirment is Win10, and I added the anaconda path to the environment variables's PATH’,the cv2.imshow worked
C:\Users\user\Anaconda3
C:\Users\user\Anaconda3\Scripts
Then restart the windows
Although this is already answered, for me conda-forge solution worked with a hack.
My workstation is a centos 6 machine, and I use conda virtual environment (anaconda 2). Create an environment
conda create --name test python=2.7
and then activate it
conda activate test
Now install opencv from conda-forge
conda install -c conda-forge opencv
Now install matplotlib in this environment (and this is hack 1)
conda install matplotlib
Let's check now imshow works or not. In a terminal, activate test environment and start python. In the interpreter, do
import cv2
import matplotlib.pyplot as plt # hack 2
img = cv2.imread('your_image_file',0)
cv2.imshow('image',img)
This should pop up a window showing image. I did not further research how this solved the case.
Note 1: You may see some error related to xkb, then in your .bashrc file add
export QT_XKB_CONFIG_ROOT=/usr/share/X11/xkb
Note 2: You may see some error related to XDG_RUNTIME_DIR, then in your .bashrc file also add
export XDG_RUNTIME_DIR=.tmp/myruntime
and define myruntime by mkdir -p .tmp/myruntime
For my system (Ubuntu 18.04) the following was working.
First:
sudo apt-get update -y
sudo apt-get install -y libgtk2.0-dev
conda create -n py35 python=3.5
conda activate py35
Then configure the environment
pip install Cython
pip install scikit-build
conda install -c anaconda cmake
pip install dlib
pip install face_recognition
pip install imutils
And finally:
pip install opencv-contrib-python
Easy with Ubuntu 18.04. It works for me:
Remove opencv-python:
pip3 uninstall opencv-python
And then re-install opencv-python:
pip3 install opencv-python
Issue was resolved.
I was able to fix it by installing a previous version of opencv-contrib-python.
I'm using Ubuntu 18

Categories

Resources