How can I install lxml dependencies on Amazon EC2 linux? - python

I tried:
sudo yum install libxml2-dev libxslt-dev python-dev
and I got:
Loaded plugins: priorities, update-motd, upgrade-helper
amzn-main/latest | 2.1 kB 00:00
amzn-updates/latest | 2.3 kB 00:00
No package libxml2-dev available.
No package libxslt-dev available.
No package python-dev available.
pip did not work either.
I got the package like this:
git clone --branch lxml-3.2.4 https://github.com/lxml/lxml
Then I tried without cython because sudo python setup.py install gave me RuntimeError: ERROR: Trying to build without Cython, but pre-generated 'src/lxml/lxml.etree.c' is not available
sudo python setup.py install --without-cython
Then I got:
sudo python setup.py install --without-cython
Building lxml version 3.2.4.
WARNING: Trying to build without Cython, but pre-generated 'src/lxml/lxml.etree.c' is not available.
WARNING: Trying to build without Cython, but pre-generated 'src/lxml/lxml.objectify.c' is not available.
Building without Cython.
ERROR: /bin/sh: xslt-config: command not found
** make sure the development packages of libxml2 and libxslt are installed **
Using build configuration of libxslt
/usr/lib64/python2.6/distutils/dist.py:266: UserWarning: Unknown distribution option: 'bugtrack_url'
warnings.warn(msg)
running install
running bdist_egg
running egg_info
writing src/lxml.egg-info/PKG-INFO
writing top-level names to src/lxml.egg-info/top_level.txt
writing dependency_links to src/lxml.egg-info/dependency_links.txt
reading manifest file 'src/lxml.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'lxml.etree.c' under directory 'src/lxml'
warning: no files found matching 'lxml.objectify.c' under directory 'src/lxml'
warning: no files found matching 'lxml.etree.h' under directory 'src/lxml'
warning: no files found matching 'lxml.etree_api.h' under directory 'src/lxml'
warning: no files found matching '*.txt' under directory 'src/lxml/tests'
warning: no files found matching '*.html' under directory 'doc'
writing manifest file 'src/lxml.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-x86_64/egg
running install_lib
running build_py
copying src/lxml/includes/lxml-version.h -> build/lib.linux-x86_64-2.6/lxml/includes
running build_ext
building 'lxml.etree' extension
gcc -pthread -fno-strict-aliasing -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -I/home/ec2-user/lxml/src/lxml/includes -I/usr/include/python2.6 -c src/lxml/lxml.etree.c -o build/temp.linux-x86_64-2.6/src/lxml/lxml.etree.o
gcc: error: src/lxml/lxml.etree.c: No such file or directory
gcc: fatal error: no input files
compilation terminated.
error: command 'gcc' failed with exit status 4
[ec2-user#ip-172-31-14-83 lxml]$
It really wants me to make sure the development packages of libxml2 and libxslt are installed.
If I can do that I hope it will work. Any ideas? Thanks! It seems people have related issues to this, but most are on other systems.
UPDATE:
Based on Vor's suggestion I used the -devel extension and that worked to install the dependencies.
sudo yum install libxml2-devel libxslt-devel python-devel
Unfortunately I get this:
$sudo python setup.py install --without-cython
Building lxml version 3.4.0.
WARNING: Trying to build without Cython, but pre-generated 'src/lxml/lxml.etree.c' is not available.
WARNING: Trying to build without Cython, but pre-generated 'src/lxml/lxml.objectify.c' is not available.
Building without Cython.
Using build configuration of libxslt 1.1.28
Building against libxml2/libxslt in the following directory: /usr/lib64
/usr/lib64/python2.6/distutils/dist.py:266: UserWarning: Unknown distribution option: 'bugtrack_url'
warnings.warn(msg)
running install
running bdist_egg
running egg_info
writing requirements to src/lxml.egg-info/requires.txt
writing src/lxml.egg-info/PKG-INFO
writing top-level names to src/lxml.egg-info/top_level.txt
writing dependency_links to src/lxml.egg-info/dependency_links.txt
reading manifest file 'src/lxml.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'lxml.etree.c' under directory 'src/lxml'
warning: no files found matching 'lxml.objectify.c' under directory 'src/lxml'
warning: no files found matching 'lxml.etree.h' under directory 'src/lxml'
warning: no files found matching 'lxml.etree_api.h' under directory 'src/lxml'
warning: no files found matching '*.html' under directory 'doc'
writing manifest file 'src/lxml.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-x86_64/egg
running install_lib
running build_py
copying src/lxml/includes/lxml-version.h -> build/lib.linux-x86_64-2.6/lxml/includes
running build_ext
building 'lxml.etree' extension
gcc -pthread -fno-strict-aliasing -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -I/usr/include/libxml2 -I/home/ec2-user/lxml/src/lxml/includes -I/usr/include/python2.6 -c src/lxml/lxml.etree.c -o build/temp.linux-x86_64-2.6/src/lxml/lxml.etree.o -w
gcc: error: src/lxml/lxml.etree.c: No such file or directory
gcc: fatal error: no input files
compilation terminated.
error: command 'gcc' failed with exit status 4

I had the same problem. After this line:
sudo yum install libxml2-devel libxslt-devel python-devel
I tried again with
sudo yum install lxml
And it worked :)

The problem on EC2 t.micro is that you run out of memory to compile the lxml package.
When you run: dmesg | tail you will see the issue
[167588.470868] Out of memory: Kill process 9804 (cc1) score 499 or sacrifice child
[167588.475046] Killed process 9804 (cc1) total-vm:693864kB, anon-rss:521696kB, file-rss:0kB
The only way is to install the python-lxml package via yum or other package manager.

sudo yum install libxml2-devel libxslt-devel python-devel
add
sudo yum install gcc
and this worked for me.

Add on to huapito's answer. Following code work for me.
sudo yum install gcc libxml2-devel libxslt-devel python-devel
sudo pip install lxml

On CentOS it is python-devel, not python-dev. Same thing with others.

The following worked for me:
sudo yum install libxml2-devel libxslt-devel python-devel
On AWS EC2 instance:
Amazon Linux AMI release 2015.09

Broken build left intermediate files in the process. try
rm -rf /tmp/pip-build-root

On my AWS machine I had both python2.7 and python3.4 and I was looking to install lxml for python3.4 - I found that I had to use
sudo yum install python34-devel

Related

Errors when installing MySQL-python package after mysql-connector-c got upgraded

Previously, my installation works normally, since it uses pip's cache directory. But when I installed a brew package, brew automatically removed mysql-connector-c, and added mysql-client 8.0.19 instead. From that moment, my application can't connect to the DB anymore:
django.core.exceptions.ImproperlyConfigured: Error loading MySQLdb module: dlopen([venv_location]/lib/python2.7/site-packages/_mysql.so, 2):
Library not loaded: /usr/local/opt/mysql-connector-c/lib/libmysqlclient.18.dylib
Referenced from: [venv_location]/lib/python2.7/site-packages/_mysql.so Reason: image not found.
It seems that the MySQL-python package still points to the old mysql-connector-c. I decided to reinstall the package without using the cache:
pip install MySQL-python==1.2.5 --no-cache-dir
But I got this error instead:
creating build/temp.macosx-10.15-x86_64-2.7
cc -fno-strict-aliasing -fno-common -dynamic -g -Os -pipe -fno-common -fno-strict-aliasing -fwrapv -DENABLE_DTRACE -DMACOSX -DNDEBUG -Wall -Wstrict-prototypes -Wshorten-64-to-32 -iwithsysroot /usr/local/libressl/include -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -I/usr/local/opt/openssl/include -arch x86_64 -pipe -Dversion_info=(1,2,5,'final',1) -D__version__=1.2.5 -I/usr/local/opt/mysql-client/include/mysql -I/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c _mysql.c -o build/temp.macosx-10.15-x86_64-2.7/_mysql.o
_mysql.c:44:10: fatal error: 'my_config.h' file not found
#include "my_config.h"
^~~~~~~~~~~~~
1 error generated.
error: command 'cc' failed with exit status 1
I have searched for this error, but the answers are usually related to brew install mysql-connector-c, while this package is not available from brew anymore. So how can I handle this exception?
Here is my /usr/local/opt (../Cellar/mysql-client/6.1.11 is invalid since it has been upgraded):
mysql-client -> ../Cellar/mysql-client/8.0.19
mysql-connector-c -> ../Cellar/mysql-client/6.1.11
Should I reinstall the old version of mysql-client to install MySQL-python from cache and keep using the old version? Or is there another way to deal with 'my_config.h' file not found?
This procedure worked for me (I'm adding the mysql-client and the mysql-connector-c removal to it):
brew remove mysql mysql-client mysql-connector-c
brew install mysql#5.7
brew link --force mysql#5.7
pip install mysql-python
I read a post and adjusted the procedure to link things in the /usr/local/bin instead of adding a new route to the PATH variable.
Reference: MacOS: "pip install MySQL-python" returning an error: "_mysql.c:44:10: fatal error: 'my_config.h' file not found"

/usr/bin/ld: cannot find -lffi while installing cffi 1.11.5 package

enter code hereI'm trying to install paramiko module in my linux server. The pip is blocked on the server, So i am manually copying the package zips and installing them from setup.py
when i try installing paramiko i get
error: Could not find suitable distribution for Requirement.parse('pynacl>=1.0.1')
so i download the PyNaCl package, but i get
distutils.errors.DistutilsError: Could not find suitable distribution for Requirement.parse('cffi>=1.1.0')
pynacl needs cffi 1.1.0 above:
so while installing cffi1.11.5 i get the below error:
[root#homecffi-1.11.5]# python3.5 setup.py install
running install
---
---
installing library code to build/bdist.linux-x86_64/egg
--
--
building '_cffi_backend' extension
gcc -pthread -fno-strict-aliasing -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -DUSE__THREAD -DHAVE_SYNC_SYNCHRONIZE -I/usr/local/lib/libffi-3.1/include -I/usr/local/include/python3.5m -c c/_cffi_backend.c -o build/temp.linux-x86_64-3.5/c/_cffi_backend.o
c/_cffi_backend.c: In function ‘b_callback’:
c/_cffi_backend.c:5911: warning: ‘ffi_prep_closure’ is deprecated (declared at /usr/local/include/ffi.h:341)
gcc -pthread -shared build/temp.linux-x86_64-3.5/c/_cffi_backend.o -L$(libdir)/../lib64 -lffi -o build/lib.linux-x86_64-3.5/_cffi_backend.cpython-35m-x86_64-linux-gnu.so
/usr/bin/ld: cannot find -lffi
collect2: ld returned 1 exit status
error: command 'gcc' failed with exit status 1
Can anyone help me with this?
If this is a Debian or Ubuntu machine,
sudo apt install libffi-dev
and then try again.
For other distributions, the installation method and package name may be different.

No OCR tool found in python

I have downloaded Mayan EDMS-Electronic Document Management System from GitHub and I configured project using Django server. I had added the required libraries based on requirement. Now the project runs with error
ocr.exceptions.OCRError: No OCR tool found
When I searched this error, I found Pyocr looks for the OCR tools (Tesseract, Cuneiform, etc) installed on your system and just tells you what it has found.
Then I tried to install tesseract using the command -->pip install tesseract-ocr.
I got this error
Requirement already satisfied: cython in ./venv2/lib/python2.7/site-packages (from tesseract-ocr) (0.28.4)
running bdist_wheel
running build
running build_py
file tesseract_ocr.py (for module tesseract_ocr) not found
file tesseract_ocr.py (for module tesseract_ocr) not found
running build_ext
building 'tesseract_ocr' extension
creating build
creating build/temp.linux-x86_64-2.7
x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fno-strict-aliasing -Wdate-time -D_FORTIFY_SOURCE=2 -g -fdebug-prefix-map=/build/python2.7-l1RrwO/python2.7-2.7.14=. -fstack-protector-strong -Wformat -Werror=format-security -fPIC -I/usr/include/python2.7 -c tesseract_ocr.cpp -o build/temp.linux-x86_64-2.7/tesseract_ocr.o
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
tesseract_ocr.cpp:600:10: fatal error: leptonica/allheaders.h: No such file or directory
#include "leptonica/allheaders.h"
please help me to solve this issue. Thanks in advance.
Tesseract is installed on the OS using the apt-get command. The command you are using (PIP) is for installing Python packages, that is the reason for the error.
For reference: http://docs.mayan-edms.com/en/stable/topics/deploying.html#deploying
If using a Debian or Ubuntu based Linux distribution, get the executable requirements using:
sudo apt-get install g++ gcc ghostscript gnupg1 graphviz libjpeg-dev libmagic1 \
libpq-dev libpng-dev libreoffice libtiff-dev poppler-utils postgresql \
python-dev python-pip python-virtualenv redis-server sane-utils supervisor \
tesseract-ocr zlib1g-dev -y

Failed to build the bsdiff module

Good afternoon all,
I am having trouble using the bsdiff module with Python. While I can use shell scripts I would prefer to have a cross-platform solution.
I have downloaded bsdiff4-1.1.4 and attempt to run setup.py as follows:
Files list:
bsdiff4 build CHANGELOG.txt do.sh examples Makefile README.rst setup.py
I run:
python3 setup.py build
And I get this error:
/usr/lib/python3.4/distutils/dist.py:260: UserWarning: Unknown distribution option: 'entry_points'
warnings.warn(msg)
running build
running build_py
running build_ext
building 'bsdiff4.core' extension
x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -g -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 -fPIC -I/usr/include/python3.4m -c bsdiff4/core.c -o build/temp.linux-x86_64-3.4/bsdiff4/core.o
bsdiff4/core.c:8:20: fatal error: Python.h: No such file or directory
#include <Python.h>
^
compilation terminated.
error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
I want to be able to use python to apply patches to binary files.
Any help appreciated.
José
You need to install apt-get install python3-dev.
ubuntu python3-dev
header files and a static library for Python
If it is your system python you will need sudo

Why can't I install greenlet (just a basic python package)?

I untarred it from: http://pypi.python.org/pypi/greenlet#downloads
(I'm in my virtual env. I sourced it, too, thats why I didn't sudo)
$ py setup.py install
/usr/lib/python2.6/distutils/dist.py:266: UserWarning: Unknown distribution option: 'repository'
warnings.warn(msg)
running install
running bdist_egg
running egg_info
writing greenlet.egg-info/PKG-INFO
writing top-level names to greenlet.egg-info/top_level.txt
writing dependency_links to greenlet.egg-info/dependency_links.txt
reading manifest file 'greenlet.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'greenlet.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-i686/egg
running install_lib
running build_ext
building 'greenlet' extension
creating build
creating build/temp.linux-i686-2.6
gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/include/python2.6 -c greenlet.c -o build/temp.linux-i686-2.6/greenlet.o
In file included from greenlet.c:5:
greenlet.h:11: fatal error: Python.h: No such file or directory
compilation terminated.
error: command 'gcc' failed with exit status 1
On Ubuntu you'd need to install the python-dev package. This provides the Python.h header:
% dpkg -S /usr/include/python2.6/Python.h
python2.6-dev: /usr/include/python2.6/Python.h
you don't have python-dev installed so there is no Python.h header.

Categories

Resources