I used
conda install gdal
to install the GDAL packages. But I had the following error when importing the packages.
>>> from osgeo import gdal
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Users/danqing0703/anaconda/lib/python2.7/site-packages/osgeo/__init__.py", line 21, in <module>
_gdal = swig_import_helper()
File "/Users/danqing0703/anaconda/lib/python2.7/site-packages/osgeo/__init__.py", line 17, in swig_import_helper
_mod = imp.load_module('_gdal', fp, pathname, description)
ImportError: dlopen(/Users/danqing0703/anaconda/lib/python2.7/site-packages/osgeo/_gdal.so, 2): Library not loaded: libgdal.20.dylib
Referenced from: /Users/danqing0703/anaconda/lib/python2.7/site-packages/osgeo/_gdal.so
Reason: image not found
>>> from osgeo import ogr
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Users/danqing0703/anaconda/lib/python2.7/site-packages/osgeo/__init__.py", line 21, in <module>
_gdal = swig_import_helper()
File "/Users/danqing0703/anaconda/lib/python2.7/site-packages/osgeo/__init__.py", line 17, in swig_import_helper
_mod = imp.load_module('_gdal', fp, pathname, description)
ImportError: dlopen(/Users/danqing0703/anaconda/lib/python2.7/site-packages/osgeo/_gdal.so, 2): Library not loaded: libgdal.20.dylib
Referenced from: /Users/danqing0703/anaconda/lib/python2.7/site-packages/osgeo/_gdal.so
Reason: image not found
What should I do to have GDAL imported in Python?
For windows users (as of December 2015):
conda install gdal
conda upgrade numpy
Installing gdal will downgrade numpy, so then upgrade it back up.
I recently had occasion to use windows for a change and I was pleasantly surprised that gdal "works" easily now.
Windows+python+gis people worldwide should be celebrating this. (that gdal-python goes in easily on windows...not that windows is one step closer to linux ;) )
I just made the mistake of executing the previously proposed command in the base environment of Conda:
conda install -c conda-forge gdal
This took ages to "solve environment" and, in the end, found numerous conflicts which halted the installation.
Given that, I instead created a separate environment with:
conda create -n gdal python=3.8
And activated it with:
conda activate gdal
And then executed the first command (as well as all others listed in the documentation). This worked fast and without any errors.
you can also use the channel conda-forge
conda install -c conda-forge gdal
as suggested on the anaconda website.
I used
conda install -c conda-forge gdal
On a Fedora 30 machine and it lead me down a path of library conflict hell!
conda install gdal
worked on my first try
The following works reliably for me Ubuntu 20.04:
conda update conda
sudo apt-get install libgdal-dev gdal-bin
export CPLUS_INCLUDE_PATH=/usr/include/gdal
export C_INCLUDE_PATH=/usr/include/gdal
# check gdal version with
gdal-config --version
echo '
name: raster-pipeline
channels:
- defaults
dependencies:
- python=3.7.*
- cython
- numpy
- ipython
- libspatialindex
- libgdal=USE_GDAL_VERSION
- gdal=USE_GDAL_VERSION
- pip
- pip:
- numpy>=1.18.5
- GDAL==USE_GDAL_VERSION
- pyproj>=2.6.1.post1
- rasterio>=1.1.5
' > raster_pipeline.yml
conda env create -f raster_pipeline.yml -v
conda activate raster_pipeline
python -c "from osgeo import gdal"
conda deactivate
This works for me
> CONDA_SUBDIR=osx-64 conda create -n my_env python=3.7
> conda activate my_env
> conda env config vars set CONDA_SUBDIR=osx-64
> conda install gdal
Now use,
> python3
> from osgeo import gdal
I had the same problem, and after several days trying different solutions, I found the problem in conflict between Anaconda version and python version! if you have both Python and Anaconda on your system, then uninstall the python and use anaconda prompt to run this:
pip install gdal
This worked for me . Hope it is useful to someone . I am using Ubuntu 20.04: It install osgeo also with gdal
Step 1 : conda install -c conda-forge gdal
If there is still some error then
Step 2 : conda config --set channel_priority strict”
Step 3 : conda update gdal
For existing conda-env, I did do like below on Ubuntu 20.04:
conda update conda
sudo apt-get install libgdal-dev gdal-bin
$export CPLUS_INCLUDE_PATH=/usr/include/gdal
$export C_INCLUDE_PATH=/usr/include/gdal
$gdal-config --version
gdal3.0.4
$whereis pip
/usr/local/anaconda3/bin/pip # anaconda's pip
$/usr/local/anaconda3/bin/pip install setuptools==57.4.0
$/usr/local/anaconda3/bin/pip install gdal==3.0.4
Related
I created a virtualenv with python 3.10 and installed open3d and PyTorch according to the instructions on open3d-ml webpage: Open3d-ML but when I tested it with import open3d.ml.torch I get the error:
Exception: Open3D was not built with PyTorch support!
Steps to reproduce
python3.10 -m venv .venv
source .venv/bin/activate
pip install --upgrade pip
pip install open3d
pip install torch torchvision torchaudio
Error
% python -c "import open3d.ml.torch as ml3d"
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Users/xx/.venv/lib/python3.10/site-packages/open3d/ml/torch/__init__.py", line 34, in <module>
raise Exception('Open3D was not built with PyTorch support!')
Exception: Open3D was not built with PyTorch support!
Environment:
% python3 --version
Python 3.10.9
% pip freeze
open3d==0.16.1
torch==1.13.1
torchaudio==0.13.1
torchvision==0.14.1
OS
macOS 12.6
Kernel Version: Darwin 21.6.0
I also checked below similar issues but they don't have answers:
https://github.com/isl-org/Open3D/discussions/5849
https://github.com/isl-org/Open3D-ML/issues/557
Open3D-ML and pytorch
According to this issue 5849 the problem can't be related only to MacOs because, in a docker with Ubuntu20.04, there is a similar error.
Does anyone know how we can tackle this?
You will need to enable PyTorch / Tensorflow when building with cmake - have a look at https://medium.com/#kidargueta/installing-open3d-ml-for-3d-computer-vision-with-pytorch-d640a6862e19 and http://www.open3d.org/docs/release/compilation.html
cmake -DBUILD_CUDA_MODULE=ON -DGLIBCXX_USE_CXX11_ABI=OFF -DBUILD_PYTORCH_OPS=ON -DBUILD_CUDA_MODULE=ON -DBUNDLE_OPEN3D_ML=ON -DOPEN3D_ML_ROOT=Open3D-ML -DBUILD_JUPYTER_EXTENSION:BOOL=ON -DBUILD_WEBRTC=ON -DOPEN3D_ML_ROOT=https://github.com/isl-org/Open3D-ML.git -DPython3_ROOT=/path/to/your/conda-env/bin/python ..
Finally, I decided to build Open3D from the source for Mac M1. I followed almost the official open3d page and thanks to this medium in one of the replies.
Build Open3d-ml with Pytorch on Mac M1
For the OS environment see the main question.
conda create --name open3d-ml-build python==3.10
conda activate open3d-ml-build
# install pytorch from pytorch.org
conda install pytorch torchvision torchaudio -c pytorch
# now clone open3d in a desired location
git clone --branch v0.16.1 git#github.com:isl-org/Open3D.git ./foo/open3d-0.16-build
cd open3d-0.16-build
mkdir build && cd build
git clone git#github.com:isl-org/Open3D-ML.git
Now make sure you are in the activeted conda env.
Build
(takes very long and a lot of memory)
Note on Mac M1 you don't have Cuda but Metal Performance Shaders (MPS) so I made CUDA Flags OFF in the cmake configuration.
which python3
>> /Users/XX/miniconda3/envs/open3d-ml-build/bin/python3
# in the build direcotry
cmake -DBUILD_CUDA_MODULE=OFF -DGLIBCXX_USE_CXX11_ABI=OFF \
-DBUILD_PYTORCH_OPS=ON -DBUILD_CUDA_MODULE=OFF \
-DBUNDLE_OPEN3D_ML=ON -DOPEN3D_ML_ROOT=Open3D-ML \
-DBUILD_JUPYTER_EXTENSION:BOOL=OFF \
-DPython3_ROOT=/Users/XX/miniconda3/envs/open3d-ml-build/bin/python3 ..
make -j$(sysctl -n hw.physicalcpu) [verbose=1]
If it fails, try it again or run it with verbose and look for fatal error.
Install
# Install pip package in the current python environment
make install-pip-package
# if error: Module Not found yapf
pip install yapf
# Create Python package in build/lib
make python-package
# Create pip wheel in build/lib
# This creates a .whl file that you can install manually.
make pip-package
sanity check
Again in the activated conda environment
# if not installed
pip install tensorboard
python3 -c "import open3d; import open3d.ml.torch"
pip freeze | grep torch
torch==1.13.1
torchaudio==0.13.1
torchvision==0.14.1
If you don't get any errors you should be good to go.
After all the required libraries are installed in appveyor, when the test starts it fails because it could not import one of the already installed libraries.
the error
python test.py
Traceback (most recent call last):
File "C:\projects\hapi\Examples\test.py", line 7, in <module>
import Hapi
File "C:\Miniconda-x64\envs\Hapi-env\lib\site-packages\hapi_nile-1.0.4-py3.9.egg\Hapi\__init__.py", line 31, in <module>
import Hapi.distparameters
File "C:\Miniconda-x64\envs\Hapi-env\lib\site-packages\hapi_nile-1.0.4-py3.9.egg\Hapi\distparameters.py", line 14, in <module>
import gdal
ModuleNotFoundError: No module named 'gdal'
Command exited with code 1
previously I solved the problem by installing the requirement from requitement.txt with a while loop but for some reason,it is not working now and I am getting an error that the system does not recognize while
while read requirement; do conda install --yes $requirement; done < requirement.txt
'while' is not recognized as an internal or external command,
operable program or batch file.
Command exited with code 1
now when I install the requirement using
- cmd: conda install --yes --file requirement.txt
somehow it couldn't import the libraries that has already been installed from the requirement.txt file
here is my appveyor.yml file
install:
- cmd: SET PATH=%CONDA_INSTALL_LOCN%;%CONDA_INSTALL_LOCN%\Scripts;%PATH%
- cmd: SET PATH=%MINICONDA%;%MINICONDA%\\Scripts;%PATH%
- cmd: conda config --set always_yes yes --set changeps1 no
- cmd: conda update -q conda
- cmd: conda info -a
# Create a new environment
- cmd: conda create -q -n Hapi-env python=%PYTHON_VERSION% --yes
- cmd: activate Hapi-env
- cmd: conda clean --tarballs -y
- cmd: conda clean --packages -y
# Install various dependencies
- cmd: conda config --add channels conda-forge
- cmd: conda install --yes --file requirement.txt
- cmd: python -V
- cmd: python setup.py build_ext --inplace
- cmd: python setup.py install
test_script:
- cmd: cd Examples/
- cmd: python test.py
and my requirement.txt file is
pip
numpy
gdal
affine
fiona
proj
pyproj
pandas
geopandas
matplotlib
python
scipy
shapely
statsmodels
rasterio
rasterstats
oasis
netCDF4
scikit-learn
scikit-image
ecmwf-api-client
joblib
Update:
I found that the while loop that iterates over the libraries in the requirement.txt file did not work before, however, the build did not crash and continued the build, but because there was no test at the time the build succeeded.
So while loops does not work in appveyor
I figured out where is the problem,
It was actually in the installation of gdal, and later than gdal 3.1.*
the import of gdal changes and the import
import gdal
does not work anymore and it is changed to
from osgeo import gdal
so for appveyor you don't need to change anything, and the only thing you need to change is to specify in the requirement.txt file which gdal version you want to install.
If you want to use the first import command install gdal<=3.1.*.
If you want to use the second import command install gdal> 3.1.*.
I installed pip and installed python for terminal use for a previous course. The course I am currently in, specified I install Anaconda. I installed Anaconda and when using JupyterLab for python and plotly, I installed the plotly-geo package. It says it is not installed though and when I try to install again it says it is installed.
Is it because I installed pip python prior? How would I remove all of pip and reinstall it for Anaconda?
Traceback (most recent call last):
File "./Desktop/Data_Visualization/p3_source/p3.py", line 25, in <module>
exponent_format=True,
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/plotly/figure_factory/_county_choropleth.py", line 622, in create_choropleth
"""
ValueError:
The create_choropleth figure factory requires the plotly-geo package.
Install using pip with:
$ pip install plotly-geo
Or, install using conda with
$ conda install -c plotly plotly-geo
The path of the site packages from your error suggests that you are executing your script with the non-anaconda version of python. I would suggest to uninstall it and only keep your anaconda distribution, as you can just create virtual environments should you need another python version
I am tying to install Fbprophet (python)
I followed all installation instructions (anaconda).
If I try to run Fbprophet in my anaconda prompt it works fine.
However, when I create a project in Visual studio 2017 I get the following error message:
Traceback (most recent call last):
File "C:\Users\dommarch\source\repos\FbProphet\FbProphet\FbProphet.py", line 1, in
import fbprophet
File "C:\Users\dommarch\AppData\Local\Continuum\anaconda2\envs\python36\lib\site-packages\fbprophet__init__.py", line 8, in
from fbprophet.forecaster import Prophet
File "C:\Users\dommarch\AppData\Local\Continuum\anaconda2\envs\python36\lib\site-packages\fbprophet\forecaster.py", line 17, in
import pystan # noqa F401
File "C:\Users\dommarch\AppData\Roaming\Python\Python36\site-packages\pystan__init__.py", line 9, in
from pystan.api import stanc, stan
File "C:\Users\dommarch\AppData\Roaming\Python\Python36\site-packages\pystan\api.py", line 13, in
import pystan._api # stanc wrapper
ImportError: DLL load failed: %1 is not a valid Win32 application.
If anyone has an idea it would be awesome.
Thanks
The version of (fb)prophet, pystan and python are important.
I managed to get it working for this stack:
CentOS: 7
Python: 3.8
GCC: 4.8.5
PyStan: 2.19.1.1
FbProphet: 0.7.1
And for this particular stack, you need GCC++ 2014 compiler, provided by this package:
centos-release-scl devtoolset-8
For Windows, using conda, python 3.8, steps are documented in this article and will summarize below:
Access anaconda using conda command in your command line
Build new virtual environment e.g. time_series, this environment will run in Python 3.8,
conda create -n time_series python=3.8
Access your environment using this command:
conda activate time_series
Install the C++ compiler:
conda install libpython m2w64-toolchain -c msys2
Installation of the dependencies of fbprophet:
conda install numpy cython -c conda-forge
conda install matplotlib scipy pandas -c conda-forge
conda install pystan -c conda-forge
conda install -c anaconda ephem
Install the library that will used for your time series forecasting environment:
pip install scikit-learn
pip install --user pmdarima
conda install -c conda-forge fbprophet
Note: Fbprophet was renamed to Prophet
https://github.com/facebook/prophet/issues/1874
So, you may need to install prophet too:
pip install pystan==2.19.1.1 prophet
or
conda install -c conda-forge prophet
I attempted the Anaconda installation for TensorFlow on my Ubuntu 12.04 LTS system, which went through, but while importing the library in Python, I came across an ImportError shown below.
I went through a solution given for a similar thread, but it didn't work for me.
Basically here's what I did for the installation:
$ conda create -n tensorflow python=2.7
$ source activate tensorflow
(tensorflow)$ conda install -c conda-forge tensorflow
(tensorflow)$ source deactivate
$ source activate tensorflow
Then from within the virtualenv I loaded Python, and tried to import tensorflow. What I got was the following error:
>>> import tensorflow
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/anirudh/anaconda/envs/tensorflow/lib/python2.7/site-packages/tensorflow/__init__.py", line 23, in <module>
from tensorflow.python import *
File "/home/anirudh/anaconda/envs/tensorflow/lib/python2.7/site-packages/tensorflow/python/__init__.py", line 48, in <module>
from tensorflow.python import pywrap_tensorflow
File "/home/anirudh/anaconda/envs/tensorflow/lib/python2.7/site-packages/tensorflow/python/pywrap_tensorflow.py", line 28, in <module>
_pywrap_tensorflow = swig_import_helper()
File "/home/anirudh/anaconda/envs/tensorflow/lib/python2.7/site-packages/tensorflow/python/pywrap_tensorflow.py", line 24, in swig_import_helper
_mod = imp.load_module('_pywrap_tensorflow', fp, pathname, description)
ImportError: /usr/lib/x86_64-linux-gnu/libstdc++.so.6: version `GLIBCXX_3.4.19' not found (required by /home/anirudh/anaconda/envs/tensorflow/lib/python2.7/site-packages/tensorflow/python/_pywrap_tensorflow.so)
How do I sort this?
You need to install libstdc++6 with some dependencies like gcc an g++, at least gcc-4.6 version:
sudo add-apt-repository ppa:ubuntu-toolchain-r/test
sudo apt-get update
sudo apt-get install gcc-4.9 g++-4.9
sudo apt-get install libstdc++6
Or you can install linux-headers and build-essential witch contains some useful tools like gcc compiler, make .... tools for compiling and building software from source.
This error probably relates to you glibc version. There are some topics regarding this:Where can I get a copy of the file libstdc++.so.6.0.15
First check whether the required version is on your system.
$ strings /usr/lib/x86_64-linux-gnu/libstdc++.so.6 | grep GLIBCXX
If it is not listed, you can try
$ sudo add-apt-repository ppa:ubuntu-toolchain-r/test
$ sudo apt-get update
$ sudo apt-get upgrade
$ sudo apt-get dist-upgrade
"sudo apt-get dist-upgrade" may not be required.
It happened for me in CentOS 7
Running python script in conda environment I got very similar error about "GLIBCXX_3.4.22..." required by tensorflow wrapper.
In my case first I did was:
"conda install libgcc"
Some libraries were installed, but that didn't helped me. I finally solved the issue by deleting "GLIBCXX" from the folder that came with Schroedinger maestro software that was installed earlier. I think, there was some sort of conflict between two libraries.