I'm trying to install the python package cvxopt via pip into a virtual environment on a Centos 6 system.
Since lapack is a prerequisite, and since I don't have superuser access to the system, I've installed lapack locally using the following:
$ cmake -DCMAKE_INSTALL_PREFIX=/home/tom .
$ make install
I notice that after this step liblapack.a is located in my local lib directory, /home/tom/lib, as expected.
However, when I activate my virtual environment and run pip install cvxopt I get the following error:
<snip>
gcc -pthread -shared
build/temp.linux-x86_64-2.7/src/C/base.o
build/temp.linux-x86_64-2.7/src/C/dense.o
build/temp.linux-x86_64-2.7/src/C/sparse.o
-L/usr/lib -L/public/apps/python/2.7.4/lib
-lm -llapack -lblas -lpython2.7 -o build/lib.linux-x86_64-2.7/cvxopt/base.so
/usr/bin/ld: cannot find -llapack
collect2: ld returned 1 exit status
error: command 'gcc' failed with exit status 1
I've tried adding /home/tom/lib to both LD_LIBRARY_PATH and LD_RUN_PATH but neither of these seem to affect the gcc command line options that pip is using to build the module.
Can anyone tell me how to give library search paths to pip? Is there a better way?
After some experimentation, I was able to solve the problem. The first thing I had to do was enable the shared libraries for LAPACK:
~/src/lapack-3.5.0 $ cmake -DBUILD_SHARED_LIBS=ON -DCMAKE_INSTALL_PREFIX=/home/tom .
~/src/lapack-3.5.0 $ make install
Then I exported LD_FLAGS to include the proper library search path:
export LD_FLAGS="-L/home/tom/lib"
Then I re-attempted the install of cvxopt from source into the virtual environment:
~/src/cvxopt-1.1.6 $ . venv/bin/activate
~/src/cvxopt-1.1.6 (venv)$ python setup.py build install
I didn't experiment with LD_LIBRARY_PATH or LD_RUN_PATH to determine whether they were still required with LD_FLAGS. However, I did determine that a shared LAPACK library was definitely required; cvxopt wouldn't link against the static library.
Related
I'm following this article to install caffe: https://qengineering.eu/install-caffe-on-ubuntu-18.04-with-opencv-4.2.html
and get this error after running make all with libboost-all-dev and libpython3 already installed.
LD -o .build_release/lib/libcaffe.so.1.0.0
/usr/bin/ld: cannot find -lboost_python3
/usr/bin/ld: cannot find -lpython3.6m
collect2: error: ld returned 1 exit status
make: *** [Makefile:596: .build_release/lib/libcaffe.so.1.0.0] Error 1
I have a feeling I may need to create a soft link for the files into path or append to my PATH, LIBRARY_PATH, or LD_LIBRARY_PATH, but I'm not actually able to locate the libpython.so or libboost files, so I'm not sure how that would be done.
First check the exact version of python that python3 is pointing to and also whether you have python3.6m
python3 -V
python3.6m -V
Both should return you Python 3.6m, otherwise find out exact version of python that is installed or version of python you want to use, accordingly change it in Makefile.
# say it's python3.7
PYTHON_LIBRARIES ?= boost_python3 python3.7
Locate libboost:
locate boost | fgrep .so
If libboost-all-dev is installed, located path will be either /usr/lib/x86_64-linux-gnu/ or /usr/lib64/ or /usr/lib/ or /usr/local/lib/ or some thing like this.
Then depending upon path you get (say you get /usr/lib/), look for exact libboost.so :
ls /usr/lib/libboost_python*.so
If return list includes libboost_python3.so, check whether LD_LIBRARY_PATH includes path to that (path here for ex: /usr/lib), otherwise do:
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/path/to/libboost/
You can add it to your ~/.bashrc:
echo 'export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/path/to/libboost/' >> ~/.bashrc
If LD_LIBRARY_PATH includes path to .so for libboost, but name is different, say it's libboost_python-py37.so, accordingly change in Makefile.
# say it's python3.7
PYTHON_LIBRARIES ?= boost_python-py37 python3.7
Then do:
make all
While trying to install cvxpy package using pip install on Mac, I get the following error message:
warning: include path for stdlibc++ headers not found; pass '-std=libc++' on the command line to use the libc++ standard library instead [-Wstdlibcxx-not-found]
In file included from cvxpy/cvxcore/src/cvxcore.cpp:15:
cvxpy/cvxcore/src/cvxcore.hpp:18:10: fatal error: 'vector' file not found
#include <vector>
^~~~~~~~
1 warning and 1 error generated.
error: command '/usr/bin/clang' failed with exit status 1
Mac is running OS Mojave.
I solved the issue using the following steps,
First I tried changing the flags to instruct the installation to use libc++,
CFLAGS=-stdlib=libc++ pip install cvxpy
Then it complained about having an invalid deployment target for -stdlib=libc++ (requires OS X 10.7 or later).
Then I ran the following command to set the deployment target,
export MACOSX_DEPLOYMENT_TARGET=10.10
Then running the first command(CFLAGS=-stdlib=libc++ pip install cvxpy) again installed cvxpy successfully.
I have been struggling with this all weekend and the most success I have found so far is installing cvxpy in an anaconda environment with these two lines:
conda install -c conda-forge lapack
conda install -c cvxgrp cvxpy
I had a similar error on Mojave. The problem is that the location of the headers installed by XCode command-line tools (which includes clang) have changed. I was able to get it working by adding this to my ~/.bash_profile and opening a new shell:
export CFLAGS="-I/usr/local/include -L/usr/local/lib -I$(brew --prefix openssl)/include -I$(xcrun --show-sdk-path)/usr/include"
This adds flags to CLANG that tell it to run the xcrun command to find the headers. It also adds the homebrew openssl headers to the clang path, which may not be necessary for this case (and assumes you have them installed).
See: https://stackoverflow.com/a/52871908/8344813
I seem to be one of the many people struggling to install gensim on windows. I have trawled through countless forums but the errors poster there never appear to match my errors. So hopefully someone can point me in the right direction!
I am running Windows Server 2012 R2 Standard 64-bit. I have installed MinGW & Anaconda 2.2.0 (64-bit), which comes with Python 2.7.9.
I have added a file distutils.cfg into C:\Users\Sam\Anaconda\Lib\distutils with the contents:
[build]
compiler=mingw32
I have added C:\MinGW\bin to my Environment variables.
If I install gensim using pip I do not get any errors, until I try to run Word2Vec when I get the error:
C:\Users\sam.passmore\AppData\Local\Continuum\Anaconda\lib\site-packages\gensim\models\word2vec.py:459: UserWarning: C extension com
pilation failed, training will be slow. Install a C compiler and reinstall gensim for fast training.
So I have uninstalled gensim and tried to re-install using the mingw32 compiler, but this gives me this error:
python setup.py build --compiler=mingw32
c:\users\sam.passmore\appdata\local\continuum\anaconda\lib\site-packages\setuptools-14.3-py2.7.egg\setuptools\dist.py:282: UserWarni
ng: Normalizing '0.11.1-1' to '0.11.1.post1'
running build
running build_ext
building 'gensim.models.word2vec_inner' extension
C:\MinGW\bin\gcc.exe -DMS_WIN64 -mdll -O -Wall -Igensim\models -IC:\Users\sam.passmore\AppData\Local\Continuum\Anaconda\include -IC:
\Users\sam.passmore\AppData\Local\Continuum\Anaconda\PC -IC:\Users\sam.passmore\AppData\Local\Continuum\Anaconda\lib\site-packages\n
umpy\core\include -c ./gensim/models/word2vec_inner.c -o build\temp.win-amd64-2.7\Release\.\gensim\models\word2vec_inner.o
gcc: error: ./gensim/models/word2vec_inner.c: No such file or directory
gcc: fatal error: no input files
compilation terminated.
command 'C:\\MinGW\\bin\\gcc.exe' failed with exit status 1
setup.py:82: UserWarning:
********************************************************************
WARNING: %s could not
be compiled. No C extensions are essential for gensim to run,
although they do result in significant speed improvements for some modules.
%s
Here are some hints for popular operating systems:
If you are seeing this message on Linux you probably need to
install GCC and/or the Python development package for your
version of Python.
Debian and Ubuntu users should issue the following command:
$ sudo apt-get install build-essential python-dev
RedHat, CentOS, and Fedora users should issue the following command:
$ sudo yum install gcc python-devel
If you are seeing this message on OSX please read the documentation
here:
http://api.mongodb.org/python/current/installation.html#osx
********************************************************************
The gensim.models.word2vec_inner extension moduleThe output above this warning shows how the compilation failed.
"The output above this warning shows how the compilation failed.")
building 'gensim.models.doc2vec_inner' extension
C:\MinGW\bin\gcc.exe -DMS_WIN64 -mdll -O -Wall -Igensim\models -IC:\Users\sam.passmore\AppData\Local\Continuum\Anaconda\include -IC:
\Users\sam.passmore\AppData\Local\Continuum\Anaconda\PC -IC:\Users\sam.passmore\AppData\Local\Continuum\Anaconda\lib\site-packages\n
umpy\core\include -c ./gensim/models/doc2vec_inner.c -o build\temp.win-amd64-2.7\Release\.\gensim\models\doc2vec_inner.o
gcc: error: ./gensim/models/doc2vec_inner.c: No such file or directory
gcc: fatal error: no input files
compilation terminated.
command 'C:\\MinGW\\bin\\gcc.exe' failed with exit status 1
setup.py:82: UserWarning:
********************************************************************
WARNING: %s could not
be compiled. No C extensions are essential for gensim to run,
although they do result in significant speed improvements for some modules.
%s
Here are some hints for popular operating systems:
If you are seeing this message on Linux you probably need to
install GCC and/or the Python development package for your
version of Python.
Debian and Ubuntu users should issue the following command:
$ sudo apt-get install build-essential python-dev
RedHat, CentOS, and Fedora users should issue the following command:
$ sudo yum install gcc python-devel
If you are seeing this message on OSX please read the documentation
here:
http://api.mongodb.org/python/current/installation.html#osx
********************************************************************
The gensim.models.doc2vec_inner extension moduleThe output above this warning shows how the compilation failed.
"The output above this warning shows how the compilation failed."
I have exhausted all options I can think of or find, so if anyone could give some advice it would be much appreciated.
I managed to solve this after using conda install for gensim, rather than pip.
conda install gensim
I am not sure what other steps I have included above have contributed to the answer, but this was the last thing I did before I no longer was getting the 'Install a C compiler and reinstall gensim for fast training.' message.
During my research in trying to solve this problem I saw that the most common methods were adding the lines
[build]
compiler=mingw32
to the distutils.cfg file as well as ensuring MinGW is in your path. Also ensuring that the MinGW bit version is the same as your python version.
I can't seem to be able to get the python ldap module installed on my OS X Mavericks 10.9.1 machine.
Kernel details:
uname -a
Darwin 13.0.0 Darwin Kernel Version 13.0.0: Thu Sep 19 22:22:27 PDT 2013; root:xnu-2422.1.72~6/RELEASE_X86_64 x86_64
I tried what was suggested here:
http://projects.skurfer.com/posts/2011/python_ldap_lion/
But when I try to use pip I get a different error
Modules/LDAPObject.c:18:10: fatal error: 'sasl.h' file not found
*#include sasl.h
I also tried what was suggested here:
python-ldap OS X 10.6 and Python 2.6
But with the same error.
I am hoping someone could help me out here.
using pieces from both #hharnisc and #mick-t answers.
pip install python-ldap \
--global-option=build_ext \
--global-option="-I$(xcrun --show-sdk-path)/usr/include/sasl"
A workaround
/usr/include appears to have moved
$ xcrun --show-sdk-path
$ sudo ln -s <the_path_from_above_command>/usr/include /usr/include
Now run pip install!
In my particular case, I couldn't simply use the pip arguments noted in other answers because I'm using it with tox to install dependencies from a requirements.txt file, and I need my tox.ini to remain compatible with non-Mac environments.
I was able to resolve this in much simpler fashion: exporting CFLAGS such that it adds an include path to the sasl headers already installed by Xcode:
$ pip install python-ldap
...
building '_ldap' extension
creating build/temp.macosx-10.10-x86_64-2.7
creating build/temp.macosx-10.10-x86_64-2.7/Modules
clang -fno-strict-aliasing -I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.10.sdk/usr/include -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -DHAVE_SASL -DHAVE_TLS -DHAVE_LIBLDAP_R -DHAVE_LIBLDAP_R -DLDAPMODULE_VERSION=2.4.19 -IModules -I/opt/openldap-RE24/include -I/usr/include/sasl -I/usr/include -I/Users/bc/.pyenv/versions/2.7.10/include/python2.7 -c Modules/LDAPObject.c -o build/temp.macosx-10.10-x86_64-2.7/Modules/LDAPObject.o
Modules/LDAPObject.c:18:10: fatal error: 'sasl.h' file not found
#include <sasl.h>
^
1 error generated.
error: command 'clang' failed with exit status 1
$ export CFLAGS="-I$(xcrun --show-sdk-path)/usr/include/sasl"
$ pip install python-ldap
...
Successfully installed python-ldap-2.4.19
Depending on whether or not you use any userspace-friendly Python tools (I use pyenv), you may have to prefix your pip commands with sudo.
I had the same problem. I'm using Macports on my Mac and I have cyrus-sasl2 installed which provides sasl.h in /opt/local/include/sasl/. You can pass options to build_ext using pip's global-option argument. To pass the include PATH to /opt/local/include/sasl/sasl.h run pip like this:
pip install python-ldap --global-option=build_ext --global-option="-I/opt/local/include/sasl"
Alternatively you could point it to whatever the output from xcrun --show-sdk-path provides. On my box that's:
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk
Then you need to determine the PATH to the sasl header files. For me that's:
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/include/sasl/
Let me know if that helps or you need a hand.
I used a combination of posts I found about this problem (including this one) to eventually come up with this (copied from a larger script):
export XC_SDK=$(xcrun --show-sdk-path)
export USR_INC=$XC_SDK/usr/include
export PATH=$USR_INC:$PATH
echo "installing python-ldap"
ARCHFLAGS=-Wno-error=unused-command-line-argument-hard-error-in-future pip install python-ldap
You can test it with python -c "import ldap"
The main reason I didn't follow the advice of #hharnisc was that on my local machine /usr/local had not moved, so I just temporarily put $XC_SDK before it on the path, and that seems to work.
some sources:
how to install PIL on Macosx 10.9?
I got this error when running buildout.
I fixed it, first finding the sasl.h file:
mdfind -name sasl.h
then defining the corresponding CFLAGS environment variable:
export CFLAGS="-I/opt/local/include/sasl"
and finally running buildout again.
I would like to install the ephem package in my Python 3.3.0 version under Windows XP.
I tried first the pip install ephem route from the system command prompt. The process aborted with the error message of Unable to find vcvarsall.bat.
I read prior advice on the net for such a difficulty. Following it I installed mingw32 in my computer, added C:\MinGW\bin to the PATH variable and provided Lib / distutils with a new file called distutils.cfg and the content:
[build]
compiler=mingw32
On running now any of the install commands (pip, *easy_install* and setup.py all work the same) the following error results:
C:\MinGW\bin\gcc.exe -mno-cygwin -mdll -O -Wall -Ilibastro-3.7.5 -IC:\Python33\
include -IC:\Python33\include -c extensions/_libastro.c -o build\temp.win32-3.3\
Release\extensions\_libastro.o
cc1.exe: error: unrecognised command line option '-mno-cygwin'
error: command 'gcc' failed with exit status 1
I have tried to understand and solve the problem, but have been unsuccesful. I have not been able even to find the place in the code where such values are established.
¿Could somebody provide any help? ¿Would ephem be one of the cases alluded by Installing Python modules in the 3.3.0 Documentation with "Not all extensions can be built with MinGW or Cygwin, but many can."?
Thanks in advance
Paco
Here is an extremely useful page of Windows binaries of python packages: http://www.lfd.uci.edu/~gohlke/pythonlibs/.
It includes binaries for pyephem.