When trying to install the python wrapper of kenlm from pip within an anaconda environment, I get the error:
(lm_1b) adamg:lm_1b adamg$ pip install https://github.com/kpu/kenlm/archive/master.zip
Collecting https://github.com/kpu/kenlm/archive/master.zip
Cache entry deserialization failed, entry ignored
Downloading https://github.com/kpu/kenlm/archive/master.zip
- 4.4MB 51.1MB/s
Installing collected packages: kenlm
Running setup.py install for kenlm ... error
Complete output from command /Users/adamg/anaconda2/envs/lm_1b/bin/python -u -c "import setuptools, tokenize;__file__='/private/var/folders/2l/hd8b8vx566ld71lfd8hjglbc0000gn/T/pip-9yzty3hd-build/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /var/folders/2l/hd8b8vx566ld71lfd8hjglbc0000gn/T/pip-aff_d2b8-record/install-record.txt --single-version-externally-managed --compile:
running install
running build
running build_ext
building 'kenlm' extension
creating build
creating build/temp.macosx-10.9-x86_64-3.6
creating build/temp.macosx-10.9-x86_64-3.6/util
creating build/temp.macosx-10.9-x86_64-3.6/lm
creating build/temp.macosx-10.9-x86_64-3.6/util/double-conversion
creating build/temp.macosx-10.9-x86_64-3.6/python
clang -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/adamg/anaconda2/envs/lm_1b/include -arch x86_64 -I/Users/adamg/anaconda2/envs/lm_1b/include -arch x86_64 -I. -I/Users/adamg/anaconda2/envs/lm_1b/include/python3.6m -c util/pool.cc -o build/temp.macosx-10.9-x86_64-3.6/util/pool.o -O3 -DNDEBUG -DKENLM_MAX_ORDER=6 -std=c++11
In file included from util/pool.cc:1:
In file included from ./util/pool.hh:4:
/Users/adamg/anaconda2/envs/lm_1b/bin/../include/c++/v1/cassert:21:10: fatal error: 'assert.h' file not found
#include <assert.h>
^~~~~~~~~~
1 error generated.
error: command 'clang' failed with exit status 1
----------------------------------------
Command "/Users/adamg/anaconda2/envs/lm_1b/bin/python -u -c "import setuptools, tokenize;__file__='/private/var/folders/2l/hd8b8vx566ld71lfd8hjglbc0000gn/T/pip-9yzty3hd-build/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /var/folders/2l/hd8b8vx566ld71lfd8hjglbc0000gn/T/pip-aff_d2b8-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /private/var/folders/2l/hd8b8vx566ld71lfd8hjglbc0000gn/T/pip-9yzty3hd-build/
The pip command works outside a conda environment, but then kenlm is not active within the environment. I was also able to run this from an AWS ec2 instance running linux, so maybe it's a Mac OSX issue. Any idea how can this be solved?
On Mac usually there is a problem on clang with different libraries that are missing. In most of this cases install Xcode will solve the problem. Also, in case this does not, you can also run in terminal xcode-select --install
Hope this helps!
Related
I want to install glpk on a raspberry pi 3. I use a virtual environment testEnv created with venv.
python3 -m venv testEnv
I have allready successfully installed pyomo
python3 -m pip install pyomo
But when I try
python3 -m pip install glpk
I get the following Error:
(testEnv) pi#raspberrypi:~/Desktop/MA-AU/Software-Test $ python3 -m pip install glpk
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Collecting glpk
Using cached https://files.pythonhosted.org/packages/ca/26/198ec4b9d1b752404a7ecb104bd1b4bfba711feaadabc0b1407de87adb26/glpk-0.4.6.tar.gz
Installing build dependencies ... done
Building wheels for collected packages: glpk
Running setup.py bdist_wheel for glpk ... error
Complete output from command /home/pi/Desktop/MA-AU/Software-Test/testEnv/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-zpb2w0ff/glpk/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /tmp/pip-wheel-0x260le1 --python-tag cp37:
running bdist_wheel
running build
running build_ext
building 'glpk' extension
creating build
creating build/temp.linux-armv7l-3.7
creating build/temp.linux-armv7l-3.7/src
arm-linux-gnueabihf-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -DVERSION_NUMBER="0.4.6" -I/home/pi/Desktop/MA-AU/Software-Test/testEnv/include -I/usr/include/python3.7m -c src/glpk.c -o build/temp.linux-armv7l-3.7/src/glpk.o
In file included from src/glpk.c:21:
src/lp.h:24:10: fatal error: glpk.h: Datei oder Verzeichnis nicht gefunden
#include <glpk.h>
^~~~~~~~
compilation terminated.
error: command 'arm-linux-gnueabihf-gcc' failed with exit status 1
----------------------------------------
Failed building wheel for glpk
Running setup.py clean for glpk
Failed to build glpk
Installing collected packages: glpk
Running setup.py install for glpk ... error
Complete output from command /home/pi/Desktop/MA-AU/Software-Test/testEnv/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-zpb2w0ff/glpk/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-record-5egd9fdm/install-record.txt --single-version-externally-managed --compile --install-headers /home/pi/Desktop/MA-AU/Software-Test/testEnv/include/site/python3.7/glpk:
running install
/tmp/pip-build-env-l3jwi8g7/lib/python3.7/site-packages/setuptools/command/install.py:37: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
setuptools.SetuptoolsDeprecationWarning,
running build
running build_ext
building 'glpk' extension
creating build
creating build/temp.linux-armv7l-3.7
creating build/temp.linux-armv7l-3.7/src
arm-linux-gnueabihf-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -DVERSION_NUMBER="0.4.6" -I/home/pi/Desktop/MA-AU/Software-Test/testEnv/include -I/usr/include/python3.7m -c src/glpk.c -o build/temp.linux-armv7l-3.7/src/glpk.o
In file included from src/glpk.c:21:
src/lp.h:24:10: fatal error: glpk.h: Datei oder Verzeichnis nicht gefunden
#include <glpk.h>
^~~~~~~~
compilation terminated.
error: command 'arm-linux-gnueabihf-gcc' failed with exit status 1
----------------------------------------
Command "/home/pi/Desktop/MA-AU/Software-Test/testEnv/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-zpb2w0ff/glpk/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-record-5egd9fdm/install-record.txt --single-version-externally-managed --compile --install-headers /home/pi/Desktop/MA-AU/Software-Test/testEnv/include/site/python3.7/glpk" failed with error code 1 in /tmp/pip-install-zpb2w0ff/glpk/
Raspberry Pi OS:
PRETTY_NAME="Raspbian GNU/Linux 10 (buster)"
NAME="Raspbian GNU/Linux"
VERSION_ID="10"
VERSION="10 (buster)"
VERSION_CODENAME=buster
ID=raspbian
ID_LIKE=debian
HOME_URL="http://www.raspbian.org/"
SUPPORT_URL="http://www.raspbian.org/RaspbianForums"
BUG_REPORT_URL="http://www.raspbian.org/RaspbianBugs"
Thanks for any help in advance!
When you get errors about missing .h files when trying to install a Python library, the reason is usually that the Python library depends on a C library, which the Python package cannot find — possibly because you have not installed it!
In this case, it looks like running apt-get install libglpk-dev should get the necessary libraries and headers for you.
This question already has an answer here:
Error on "pip install -U channels" command using for otree (Running setup.py bdist_wheel for twisted ... error)
(1 answer)
Closed 4 years ago.
I am running virtualenv with python 3.6 and am trying to load test a django app. I get the following error after all dependencies install successfully before it.
Installing collected packages: greenlet, gevent, locustio
Running setup.py install for greenlet ... error
Complete output from command /Library/Frameworks/Python.framework/Versions/3.6/bin/python3.6 -u -c "import setuptools, tokenize;__file__='/private/var/folders/3t/_nx5hykn0hsfs5v1x_svhdym0000gn/T/pip-install-lrm7ibd5/greenlet/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /private/var/folders/3t/_nx5hykn0hsfs5v1x_svhdym0000gn/T/pip-record-77exz0iu/install-record.txt --single-version-externally-managed --compile:
running install
running build
running build_ext
building 'greenlet' extension
creating build
creating build/temp.macosx-10.6-intel-3.6
gcc -fno-strict-aliasing -Wsign-compare -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -arch i386 -arch x86_64 -g -I/Library/Frameworks/Python.framework/Versions/3.6/include/python3.6m -c greenlet.c -o build/temp.macosx-10.6-intel-3.6/greenlet.o
xcrun: error: invalid active developer path (/Library/Developer/CommandLineTools), missing xcrun at: /Library/Developer/CommandLineTools/usr/bin/xcrun
error: command 'gcc' failed with exit status 1
----------------------------------------
Command "/Library/Frameworks/Python.framework/Versions/3.6/bin/python3.6 -u -c "import setuptools, tokenize;__file__='/private/var/folders/3t/_nx5hykn0hsfs5v1x_svhdym0000gn/T/pip-install-lrm7ibd5/greenlet/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /private/var/folders/3t/_nx5hykn0hsfs5v1x_svhdym0000gn/T/pip-record-77exz0iu/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /private/var/folders/3t/_nx5hykn0hsfs5v1x_svhdym0000gn/T/pip-install-lrm7ibd5/greenlet/
run in terminal xcode-select --install this will install the xcode comand line tools (not xcode itself), you will be asked for a password and to agree to a license.
This happens because one (or more) of the packages include C extensions that need compilation, and to compile the xcode command line tools are needed.
A separate solution would be to install Anaconda python, miniconda if you don't need packages for scientific python, this python distribution includes a package manager (conda) that work much like pip but with the advantage of having precompiled packages
When I tried to install httrack in Ubuntu 16.04 I was not able to get those packages:
pip3 install httrack-py
Collecting httrack-py
Using cached https://files.pythonhosted.org/packages/58/b8/9499cd45ffb4efefff06090f04111c6572e7947eb4485d3333edc9505ed0/httrack-py-0.6.1.tar.gz
Installing collected packages: httrack-py
Running setup.py install for httrack-py ... error
Complete output from command /usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-96j_4z55/httrack-py/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-f9__5p13-record/install-record.txt --single-version-externally-managed --compile:
running install
running build
running build_py
creating build
creating build/lib.linux-x86_64-3.5
copying httrack.py -> build/lib.linux-x86_64-3.5
running build_ext
building 'httracklib' extension
creating build/temp.linux-x86_64-3.5
creating build/temp.linux-x86_64-3.5/src
x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -g -fdebug-prefix-map=/build/python3.5-6tVwKN/python3.5-3.5.2=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/home/petri/downloads/httrack-source -I/home/petri/downloads/httrack-source/src -I/usr/include -I/usr/include/python3.5 -I/usr/include/python3.5m -c src/httrack-py.c -o build/temp.linux-x86_64-3.5/src/httrack-py.o
src/httrack-py.c:45:29: fatal error: httrack-library.h: No such file or directory
#include "httrack-library.h"
^
compilation terminated.
error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
----------------------------------------
Command "/usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-96j_4z55/httrack-py/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-f9__5p13-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-build-96j_4z55/httrack-py/
You are using pip version 9.0.1, however version 10.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
Run the code:
sudo apt-get update
And if its not working
Try to update repository in software centre.
The pip log says:
#include "httrack-library.h"
A necessary library header is missing.
Try running sudo apt install libhttrack-dev and then pip install again.
Currently trying to run pip install python-lzo in any environment fails for me (I have tried on three different machines: ubuntu 16.04, macosx and inside a docker container based on python:2.7):
Collecting python-lzo
Downloading python-lzo-1.11.tar.gz
Building wheels for collected packages: python-lzo
Running setup.py bdist_wheel for python-lzo ... error
Complete output from command /usr/local/opt/python/bin/python2.7 -u -c "import setuptools, tokenize;__file__='/private/var/folders/2m/0shxv04j04lcm1c0dddyf4qw0000gn/T/pip-build-iOtvF1/python-lzo/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /var/folders/2m/0shxv04j04lcm1c0dddyf4qw0000gn/T/tmp4XTk5Ipip-wheel- --python-tag cp27:
/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py:251: UserWarning: 'licence' distribution option is deprecated; use 'license'
warnings.warn(msg)
running bdist_wheel
running build
running build_ext
building 'lzo' extension
creating build
creating build/temp.macosx-10.11-x86_64-2.7
clang -fno-strict-aliasing -fno-common -dynamic -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/usr/include/lzo -I/usr/local/include -I/usr/local/opt/openssl/include -I/usr/local/opt/sqlite/include -I/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c lzomodule.c -o build/temp.macosx-10.11-x86_64-2.7/lzomodule.o
lzomodule.c:35:10: fatal error: 'lzo1x.h' file not found
#include <lzo1x.h>
^
1 error generated.
error: command 'clang' failed with exit status 1
----------------------------------------
Failed building wheel for python-lzo
Running setup.py clean for python-lzo
Failed to build python-lzo
Installing collected packages: python-lzo
Running setup.py install for python-lzo ... error
Complete output from command /usr/local/opt/python/bin/python2.7 -u -c "import setuptools, tokenize;__file__='/private/var/folders/2m/0shxv04j04lcm1c0dddyf4qw0000gn/T/pip-build-iOtvF1/python-lzo/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /var/folders/2m/0shxv04j04lcm1c0dddyf4qw0000gn/T/pip-iXLv5I-record/install-record.txt --single-version-externally-managed --compile:
/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py:251: UserWarning: 'licence' distribution option is deprecated; use 'license'
warnings.warn(msg)
running install
running build
running build_ext
building 'lzo' extension
creating build
creating build/temp.macosx-10.11-x86_64-2.7
clang -fno-strict-aliasing -fno-common -dynamic -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/usr/include/lzo -I/usr/local/include -I/usr/local/opt/openssl/include -I/usr/local/opt/sqlite/include -I/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c lzomodule.c -o build/temp.macosx-10.11-x86_64-2.7/lzomodule.o
lzomodule.c:35:10: fatal error: 'lzo1x.h' file not found
#include <lzo1x.h>
^
1 error generated.
error: command 'clang' failed with exit status 1
----------------------------------------
Command "/usr/local/opt/python/bin/python2.7 -u -c "import setuptools, tokenize;__file__='/private/var/folders/2m/0shxv04j04lcm1c0dddyf4qw0000gn/T/pip-build-iOtvF1/python-lzo/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /var/folders/2m/0shxv04j04lcm1c0dddyf4qw0000gn/T/pip-iXLv5I-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /private/var/folders/2m/0shxv04j04lcm1c0dddyf4qw0000gn/T/pip-build-iOtvF1/python-lzo/
Really I am trying to build a docker container that runs this:
RUN pip install python-lzo
Any thoughts here? Thanks!
You must install liblzo2-dev in debian base image and install lzo-devel in centos base image.
Fixed this with the following Dockerfile:
FROM ubuntu:16.04
RUN apt-get update && \
apt-get install -y python python-pip python-lzo zlib1g-dev unzip
RUN pip install bx-python
So apparently you can install python-lzo directly through apt instead of using pip? sigh
I have been trying to install openfst python library for the last week, however I am stuck. I have read all similar questions on stack overflow and other websites but none of the instructions work.
I have the latest Xcode installed, using
brew install openfst
I also installed openfst, however when I want to install the python library by writing:
pip install openfst
in the terminal, I get:
Collecting openfst
Using cached openfst-1.5.0.tar.gz
Building wheels for collected packages: openfst
Running setup.py bdist_wheel for openfst
Complete output from command /Users/ali/anaconda/bin/python -c "import setuptools;__file__='/private/var/folders/36/0m4j84pd49l55mvcqmbqt3z00000gn/T/pip-build-Jqe8Nu/openfst/setup.py';exec(compile(open(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" bdist_wheel -d /var/folders/36/0m4j84pd49l55mvcqmbqt3z00000gn/T/tmpFNyllkpip-wheel-:
running bdist_wheel
running build
running build_ext
building 'fst' extension
creating build
creating build/temp.macosx-10.5-x86_64-2.7
gcc -fno-strict-aliasing -I/Users/ali/anaconda/include -arch x86_64 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/ali/anaconda/include/python2.7 -c fst.cc -o build/temp.macosx-10.5-x86_64-2.7/fst.o -std=c++11 -Wno-unneeded-internal-declaration -Wno-unused-function
In file included from fst.cc:241:
/usr/local/include/fst/util.h:24:10: fatal error: 'unordered_map' file not found
#include <unordered_map>
^
1 error generated.
error: command 'gcc' failed with exit status 1
----------------------------------------
Failed building wheel for openfst
Failed to build openfst
Installing collected packages: openfst
Running setup.py install for openfst
Complete output from command /Users/ali/anaconda/bin/python -c "import setuptools, tokenize;__file__='/private/var/folders/36/0m4j84pd49l55mvcqmbqt3z00000gn/T/pip-build-Jqe8Nu/openfst/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /var/folders/36/0m4j84pd49l55mvcqmbqt3z00000gn/T/pip-oi7XrR-record/install-record.txt --single-version-externally-managed --compile:
running install
running build
running build_ext
building 'fst' extension
gcc -fno-strict-aliasing -I/Users/ali/anaconda/include -arch x86_64 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/ali/anaconda/include/python2.7 -c fst.cc -o build/temp.macosx-10.5-x86_64-2.7/fst.o -std=c++11 -Wno-unneeded-internal-declaration -Wno-unused-function
In file included from fst.cc:241:
/usr/local/include/fst/util.h:24:10: fatal error: 'unordered_map' file not found
#include <unordered_map>
^
1 error generated.
error: command 'gcc' failed with exit status 1
----------------------------------------
Command "/Users/ali/anaconda/bin/python -c "import setuptools, tokenize;__file__='/private/var/folders/36/0m4j84pd49l55mvcqmbqt3z00000gn/T/pip-build-Jqe8Nu/openfst/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /var/folders/36/0m4j84pd49l55mvcqmbqt3z00000gn/T/pip-oi7XrR-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /private/var/folders/36/0m4j84pd49l55mvcqmbqt3z00000gn/T/pip-build-Jqe8Nu/openfst
Could someone please help me?
I would suggest compiling OpenFST from source with Python enabled. It's pretty easy:
wget http://www.openfst.org/twiki/pub/FST/FstDownload/openfst-1.7.2.tar.gz
tar zxvf openfst-1.7.2.tar.gz
cd openfst-1.7.2
./configure --enable-python
make
sudo make install