I am trying to import Rotated Mask R-CNN libraries into google colab but seem unable to do so, likely due to incompatibility issues with other libraries.
If we try the default install the following errors occur:
#Install rotated_maskRCNN
!pip install git+https://github.com/mrlooi/rotated_maskrcnn
Collecting git+https://github.com/mrlooi/rotated_maskrcnn
Cloning https://github.com/mrlooi/rotated_maskrcnn to /tmp/pip-req-build-tciz12ki
Running command git clone -q https://github.com/mrlooi/rotated_maskrcnn /tmp/pip-req-build-tciz12ki
Building wheels for collected packages: maskrcnn-benchmark
Building wheel for maskrcnn-benchmark (setup.py) ... error
ERROR: Failed building wheel for maskrcnn-benchmark
Running setup.py clean for maskrcnn-benchmark
Failed to build maskrcnn-benchmark
Installing collected packages: maskrcnn-benchmark
Running setup.py install for maskrcnn-benchmark ... error
ERROR: Command errored out with exit status 1: /usr/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-req-build-tciz12ki/setup.py'"'"'; __file__='"'"'/tmp/pip-req-build-tciz12ki/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-onk26ne6/install-record.txt --single-version-externally-managed --compile Check the logs for full command output.
When we simply execute the following two lines of code in Google Colab the following error appears:
!git clone https://github.com/mrlooi/rotated_maskrcnn.git
!python "/content/rotated_maskrcnn/setup.py" build develop
2 errors detected in the compilation of "/content/rotated_maskrcnn/maskrcnn_benchmark/csrc/cuda/deform_pool_cuda.cu".
error: command '/usr/local/cuda/bin/nvcc' failed with exit status 1
But during the install, many more errors occurred. According to the install.MD we need the following:
# follow PyTorch installation in https://pytorch.org/get-started/locally/
# we give the instructions for CUDA 9.0
conda install -c pytorch pytorch-nightly torchvision cudatoolkit=9.0
Which seems complicated to achieve.
I have ran the following code to uninstall existing cuda's and install cuda 9.0 which does seem to work, although CUDA 10.2 and 11.0 remain in usr/local/
#Uninstall the current CUDA version
!apt-get --purge remove cuda nvidia* libnvidia-*
!dpkg -l | grep cuda- | awk '{print $2}' | xargs -n1 dpkg --purge
!apt-get remove cuda-*
!apt autoremove
!apt-get update
#Install CUDA 9.0
!wget https://developer.nvidia.com/compute/cuda/9.0/Prod/local_installers/cuda-repo-ubuntu1604-9-0-local_9.0.176-1_amd64-deb
!dpkg -i cuda-repo-ubuntu1604-9-0-local_9.0.176-1_amd64-deb
!apt-key add /var/cuda-repo-9-0-local/7fa2af80.pub
!apt-get update
!apt-get install cuda=9.0.176-1
#We need CUDA 9 for maskRCNN to work
!cat /usr/local/cuda/version.txt
CUDA Version 9.0.176
Still the same error messages are maintaining and I am unsure where everything is going haywire. I have also tried to setup an anaconda environment in the Colab environment, but this defeats the purpose of using google Colab, and also didn't get this to work correctly.
Related
I am facing an issue when trying to install shop in a python virtual environment. In particular when
pip install shap
the following error appears:
ERROR: Command errored out with exit status 1:
command: /Users/federiconutarelli/Desktop/Riccaboni_python/venv/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/86/wyxlz8f56nq997hwmvpnt_km0000gn/T/pip-install-snwg2_4s/shap_4dbdc5c8e74a45cca2a3aa06af49ba31/setup.py'"'"'; __file__='"'"'/private/var/folders/86/wyxlz8f56nq997hwmvpnt_km0000gn/T/pip-install-snwg2_4s/shap_4dbdc5c8e74a45cca2a3aa06af49ba31/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /private/var/folders/86/wyxlz8f56nq997hwmvpnt_km0000gn/T/pip-wheel-nnajc046
cwd: /private/var/folders/86/wyxlz8f56nq997hwmvpnt_km0000gn/T/pip-install-snwg2_4s/shap_4dbdc5c8e74a45cca2a3aa06af49ba31/
Complete output (224 lines):
The nvcc binary could not be located in your $PATH. Either add it to your path, or set $CUDAHOME to enable CUDA
Error building cuda module: TypeError('cannot unpack non-iterable NoneType object')
WARNING: Could not compile cuda extensions
running bdist_wheel
running build
running build_py
creating build
creating build/lib.macosx-10.14.6-x86_64-3.8
creating build/lib.macosx-10.14.6-x86_64-3.8/shap
copying shap/datasets.py -> build/lib.macosx-10.14.6-x86_64-3.8/shap
copying shap/__init__.py -> build/lib.macosx-10.14.6-x86_64-3.8/shap
copying shap/links.py -> build/lib.macosx-10.14.6-x86_64-3.8/shap ...
(it is longer than that but I cut it for readability). Also, conda installation gets stuck at executing transaction. As far as I have understood, the problem lies on the fact that Shapiros depends on CUDA. However also after setting CUDA to the path
export CUDA_HOME=/usr/local/cuda-8.0
export LD_LIBRARY_PATH=${CUDA_HOME}/lib64
PATH=${CUDA_HOME}/bin:${PATH}
export PATH
the error remains. I am working on a python virtual environment. I know that anaconda should provide CUDA but anaconda seems to get stuck as well (every command get sucked at "Solving Environment".
How can I solve the issue?
Thank you all.
Try install python3 dev.
For ubuntu
sudo apt -y install python3-dev
For Centos
sudo yum -y install python3-devel
Edit: If you are using a specific version of python
sudo apt -y install python3.7-dev
Make sure that the dev version and python version are the same
I've tried to type this command:
pip3 install --user psycopg2 psycopg2-binary
Bash answers:
ERROR: running install
It appears you are missing some prerequisite to build the package from source.
You may install a binary package by installing 'psycopg2-binary' from PyPI.
If you want to install psycopg2 from source, please install the packages
required for the build and try again.
For further information please check the 'doc/src/install.rst' file (also at
<http://initd.org/psycopg/docs/install.html>).
error: command 'gcc' failed with exit status 1
----------------------------------------
ERROR: Command "/Library/Frameworks/Python.framework/Versions/3.7/bin/python3 -u -c 'import setuptools, tokenize;file='"'"'/private/var/folders/qf/kmd2_y0j2p9_10zkd5ctvm880000gn/T/pip-install-i0kg4ysj/psycopg2/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' install --record /private/var/folders/qf/kmd2_y0j2p9_10zkd5ctvm880000gn/T/pip-record-lkzzuu55/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /private/var/folders/qf/kmd2_y0j2p9_10zkd5ctvm880000gn/T/pip-install-i0kg4ysj/psycopg2/
You're not being able to install psycopg2 as the gcc compiler in your device is either outdated and not compatible to install and run it, or it does not exist.
if you just run pip install psycopg2-binary, then it will work on your development and testing environment, however it is not recommended for production.
To install psycopg2, make sure that you have gcc installed in your device by running-
gcc --version
gcc where
If it is not installed, then install it by referring to the guide here-
Installing GCC in Mac
I hope this helps. :)
after successfully installed python and pip, this command should work:
pip install psycopg2
Cloned the repo : https://github.com/RasaHQ/rasa_nlu.git
$ pip install -r requirements.txt
Building wheel for httptools (setup.py) ... error ERROR: Complete
output from command /Users/mohit/anaconda3/envs/condapy36/bin/python
-u -c 'import setuptools, tokenize;file='"'"'/private/var/folders/9t/0qgpwgy906z_sww71dnkgkr80000gn/T/pip-install-7hmlhuo6/httptools/setup.py'"'"';f=getattr(tokenize,
'"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"',
'"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))'
bdist_wheel -d
/private/var/folders/9t/0qgpwgy906z_sww71dnkgkr80000gn/T/pip-wheel-u02mzs5m
--python-tag cp36: ERROR: running bdist_wheel
error: command 'gcc' failed with exit status 1
---------------------------------------- ERROR: Failed building wheel for httptools Running setup.py clean for httptools Building
wheel for ujson (setup.py) ... error
$ pip install -e .
Obtaining file:///Users/mohit/work/research/NLP/Rasa/rasa_nlu ERROR:
Error installing
'file:///Users/mohit/work/research/NLP/Rasa/rasa_nlu': editable mode
is not supported for pyproject.toml-style projects. pip is processing
this project as pyproject.toml-style because it has a pyproject.toml
file. Since the project has a setup.py and the pyproject.toml has no
"build-backend" key for the "build_system" value, you may pass
--no-use-pep517 to opt out of pyproject.toml-style processing. See PEP 517 for details on pyproject.toml-style projects.
OS : Mac OS High Sierra
Python 3.6.8 :: Anaconda custom (64-bit)
pip 19.1
That's was actually in issue in pip (https://github.com/pypa/pip/issues/6375).
Please install using pip install --no-use-pep517 -e . or upgrade pip to the latest version.
check for visual studio, redists, vc++ and allied syncing up
This question already has answers here:
Error installing Turicreate on ubuntu python 3.8.2 on pip
(2 answers)
Closed 2 years ago.
Hello Last few days i am trying to run one script for creating the Modal for machine learning. but I am not able to install the Turicreate package in Pycharm editor.
I follow these steps to run the script.
Download Python 3.7 in this link:
(https://www.python.org/downloads/)
Download the Pycharm Editor(https://www.jetbrains.com/pycharm/)
After that, I set the project interpreter in latest python3.7 and trying to install turicreate but always I got error.
Collecting turicreate
Using cached https://files.pythonhosted.org/packages/db/54/167837569bcb816b3fe68f003f18d07ab9d5ac31b2b12b8f9b07b1ccc7a4/turicreate-4.2.tar.gz
Building wheels for collected packages: turicreate
Running setup.py bdist_wheel for turicreate: started
Running setup.py bdist_wheel for turicreate: finished with status 'error'
Complete output from command /Users/tikam/MLTikam1/venv/bin/python -u -c "import setuptools, tokenize;file='/private/var/folders/rh/qx_0gvzn6kzbqjvvbbh66t080000gn/T/pycharm-packaging1/turicreate/setup.py';f=getattr(tokenize, 'open', open)(file);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, file, 'exec'))" bdist_wheel -d /private/var/folders/rh/qx_0gvzn6kzbqjvvbbh66t080000gn/T/pip-wheel-qez5g2v_ --python-tag cp37:
running bdist_wheel
running build
installing to build/bdist.macosx-10.9-x86_64/wheel
running install
==================================================================================
ERROR
If you see this message, pip install did not find an available binary package
for your system. Supported platforms are:
* Linux x86_64 (including WSL on Windows 10).
* macOS 10.12+ x86_64.
* Python 2.7, 3.5, or 3.6.
Other possible causes of this error are:
* Outdated pip version (try `pip install -U pip`).
==================================================================================
Running setup.py clean for turicreate
Failed to build turicreate
Installing collected packages: turicreate
Running setup.py install for turicreate: started
Running setup.py install for turicreate: finished with status 'error'
Complete output from command /Users/tikam/MLTikam1/venv/bin/python -u -c "import setuptools, tokenize;file='/private/var/folders/rh/qx_0gvzn6kzbqjvvbbh66t080000gn/T/pycharm-packaging1/turicreate/setup.py';f=getattr(tokenize, 'open', open)(file);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, file, 'exec'))" install --record /private/var/folders/rh/qx_0gvzn6kzbqjvvbbh66t080000gn/T/pip-record-kjt0p8as/install-record.txt --single-version-externally-managed --compile --install-headers /Users/tikam/MLTikam1/venv/include/site/python3.7/turicreate:
running install
==================================================================================
ERROR
If you see this message, pip install did not find an available binary package
for your system. Supported platforms are:
* Linux x86_64 (including WSL on Windows 10).
* macOS 10.12+ x86_64.
* Python 2.7, 3.5, or 3.6.
Other possible causes of this error are:
* Outdated pip version (try `pip install -U pip`).
==================================================================================
----------------------------------------
Failed building wheel for turicreate
Command "/Users/tikam/MLTikam1/venv/bin/python -u -c "import setuptools, tokenize;file='/private/var/folders/rh/qx_0gvzn6kzbqjvvbbh66t080000gn/T/pycharm-packaging1/turicreate/setup.py';f=getattr(tokenize, 'open', open)(file);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, file, 'exec'))" install --record /private/var/folders/rh/qx_0gvzn6kzbqjvvbbh66t080000gn/T/pip-record-kjt0p8as/install-record.txt --single-version-externally-managed --compile --install-headers /Users/tikam/MLTikam1/venv/include/site/python3.7/turicreate" failed with error code 1 in /private/var/folders/rh/qx_0gvzn6kzbqjvvbbh66t080000gn/T/pycharm-packaging1/turicreate/
Please suggest what step i need to follow to install the turicreat package.
I want to run this script:
Script **************
import turicreate as tc
data = tc.SFrame('photoLabel.sframe')
model = tc.image_classifier.create(data, target='photoLabel')
predictions = model.predict(data)
model.export_coreml('MyClassifier.mlmodel')
It is not supported Python 3.7 right now. So you have to return to the previous Python version.
brew unlink python
brew install https://raw.githubusercontent.com/Homebrew/homebrew-core/f2a764ef944b1080be64bd88dca9a1d80130c558/Formula/python.rb
Source:https://github.com/apple/turicreate/issues/788
Or you can create virtual env to run turicreate on
Download the Python3.6 tgz file from the official website (eg. Python-3.6.6.tgz)
Unpack it with tar -xvzf Python-3.6.6.tgz
cd Python-3.6.6
run ./configure
run make altinstall to install it
(install vs altinstall explanation here Difference in details between "make install" and "make altinstall")
You'll normally find your new python install under /usr/local/bin. Now you can create a new virtualenv specifying the python version with:
virtualenv --python=python3.6 env3.6
Get into the virtualenv running the command source env3.6/source/bin/activate.
Install turicreate with the classic pip install turicreate
source:https://github.com/tensorflow/tensorflow/issues/17022
I'm having difficulties with the --global-option and --install-option settings for a requirements.txt file. Specifying the options for one library is causing other libraries installs to fail.
I'm trying to install Python libraries "grab" and "pycurl". I need to specify that pycurl be installed with option: "--with-nss". I can replicate the error on a completely clean virtual enviroment.
On a new virtual environment With requirements.txt containing:
grab==0.6.25
pycurl==7.43.0 --install-option='--with-nss'
Then installing with:
pip install -r requirements.txt
The following errors will occur.
Installing collected packages: lxml, pycurl, pytils, six, user-agent, weblib, selection, grab
Running setup.py install for lxml ... done
Running setup.py install for pycurl ... done
Running setup.py install for pytils ... error
Complete output from command /home/ec2-user/test/env/bin/python2.7 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-8GvFzA/pytils/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n
'), __file__, 'exec'))" install --record /tmp/pip-BCG3Wl-record/install-record.txt --single-version-externally-managed --compile --install-headers /home/ec2-user/test/env/include/site/python2.7/pytils --with-nss:
usage: -c [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: -c --help [cmd1 cmd2 ...]
or: -c --help-commands
or: -c cmd --help
error: option --with-nss not recognized
----------------------------------------
Command "/home/ec2-user/test/env/bin/python2.7 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-8GvFzA/pytils/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))"
install --record /tmp/pip-BCG3Wl-record/install-record.txt --single-version-externally-managed --compile --install-headers /home/ec2-user/test/env/include/site/python2.7/pytils --with-nss" failed with error code 1 in /tmp/pip-build-8GvF
zA/pytils/
My best guess at the root cause is that the option "--with-nss" is being passed to all libraries that require pycurl, and preventing install. The pytils installation fails even though the pycurl install works fine.
Is there anyway to only pass the install options to the one library?
I'm setting this up on an Amazon Elastic Beanstalk instance, so there is no option to manually run each line of the requirements.txt file - the whole install gets run at start up of the application.
Sources for --global-option and --install-option (which I think shouldn't do this):
How to maintain pip install options in requirements file made by pip freeze?
https://github.com/pypa/pip/blob/develop/docs/reference/pip_install.rst#id28
Your problem comes from the fact that PIP version on EC2 with EB is quite old and does not understand your options.
Update pip lib to latest available version with EB commands:
project_dir/.ebextensions/02-python.config:
...
commands:
01_upgrade_pip_for_venv:
command: "/opt/python/run/venv/bin/pip install --upgrade pip"
...
Now you can leave options in requirement.txt since new version of pip will be able to work with it.
project_dir/requirements.txt:
...
pycurl==7.43.0 --global-option="--with-nss"
...
(This may be redundant) Set option in EB console user interfaces or by eb CLI with command:
eb setenv PYCURL_SSL_LIBRARY=nss
Push changes to repository and Rebuild. You may have errors since execution is controlled from external scope and started with old version of PIP. Entry point of execution is outside of app on EC2 instance so I'm not sure how to bring solution that would work from scope of hooks on first deployment... But all you have to do is to deploy again, and it will use proper version of PIP, so it will work from now on, till next rebuild...