I'm getting error installing Scrapy on my ubuntu box. I'm using pip to install Scrapy. I'm aware that it needs setuptools to be installed. I got that installed using the script provided in setuptools website.
reading manifest file 'Twisted.egg-info/SOURCES.txt'
writing manifest file 'Twisted.egg-info/SOURCES.txt'
creating build/lib.linux-x86_64-2.7/twisted/internet/iocpreactor/iocpsupport
copying twisted/internet/iocpreactor/iocpsupport/iocpsupport.c -> build/lib.linux-x86_64-2.7/twisted/internet/iocpreactor/iocpsupport
copying twisted/internet/iocpreactor/iocpsupport/winsock_pointers.c -> build/lib.linux-x86_64-2.7/twisted/internet/iocpreactor/iocpsupport
copying twisted/test/raiser.c -> build/lib.linux-x86_64-2.7/twisted/test
copying twisted/runner/portmap.c -> build/lib.linux-x86_64-2.7/twisted/runner
copying twisted/python/sendmsg.c -> build/lib.linux-x86_64-2.7/twisted/python
running build_ext
x86_64-linux-gnu-gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/include/python2.7 -c conftest.c -o conftest.o
building 'twisted.runner.portmap' extension
creating build/temp.linux-x86_64-2.7
creating build/temp.linux-x86_64-2.7/twisted
creating build/temp.linux-x86_64-2.7/twisted/runner
x86_64-linux-gnu-gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/include/python2.7 -c twisted/runner/portmap.c -o build/temp.linux-x86_64-2.7/twisted/runner/portmap.o
twisted/runner/portmap.c:10:20: fatal error: Python.h: No such file or directory
#include <Python.h>
^
compilation terminated.
error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
----------------------------------------
Cleaning up...
Command /usr/bin/python -c "import setuptools, tokenize;__file__='/tmp/pip_build_root/Twisted/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-4QNuNV-record/install-record.txt --single-version-externally-managed --compile failed with error code 1 in /tmp/pip_build_root/Twisted
Storing debug log for failure in /root/.pip/pip.log
Any Idea where the process goes wrong? I've got gcc and g++ installed already.
From the error fatal error: Python.h: No such file or directory it looks like python development headers are not installed. Try this command and then try to install again.
sudo apt-get install python-dev
For installing libevent library apply this command,
sudo apt-get install libevent-dev
I figured this out on ubuntu 14.04 by (A) actually following the scrapy docs for installing on ubuntu, then (B) an error that a lot of others were getting, by following a S/O solution Error while starting new scrapy project
(A) http://doc.scrapy.org/en/1.0/topics/ubuntu.html#topics-ubuntu
then
(B)
sudo pip install pyasn1 --upgrade
Right after installing Fedora 23 that's what I did (may works on Ubuntu):
[root#x ~]# pip install scrapy
[root#x ~]# dnf install python-cffi
[root#x ~]# dnf install openssl-devel
[root#x ~]# dnf install gcc
[root#x ~]# dnf install redhat-rpm-config
[root#x ~]# dnf install libxml
[root#x ~]# dnf install libxml2-devel
[root#x ~]# dnf install libxml-devel
[root#x ~]# dnf install glib2-devel gnet2-devel
[root#x ~]# dnf install libxslt-devel
sudo apt install python3-scrapy
I have tried this ..and it worked for me.
(You should have the ssh installed)
Related
I followed the instructions and ran the following to commands to install pykaldi:
git clone https://github.com/pykaldi/pykaldi.git
cd pykaldi
sudo apt-get install autoconf automake cmake curl g++ git graphviz libatlas3-base libtool make pkg-config subversion unzip wget zlib1g-dev
sudo apt install intel-mkl-64bit-2020.4-912
python3.7 -m pip install --upgrade pip setuptools
python3.7 -m pip install numpy pyparsing ninja==1.10.0
cd tools
sudo torify ./check_dependencies.sh /usr/bin/python3.7
sudo torify ./install_protobuf.sh /usr/bin/python3.7
sudo torify ./install_clif.sh /usr/bin/python3.7
sudo torify ./install_kaldi.sh
cd ..
python3.7 -m pip install setuptools
sudo apt-get install ninja-build
sudo python3.7 setup.py install
when it comes to the last line, I get the following error:
running install
running bdist_egg
running egg_info
writing pykaldi.egg-info/PKG-INFO
writing dependency_links to pykaldi.egg-info/dependency_links.txt
writing requirements to pykaldi.egg-info/requires.txt
writing top-level names to pykaldi.egg-info/top_level.txt
reading manifest file 'pykaldi.egg-info/SOURCES.txt'
adding license file 'LICENSE'
writing manifest file 'pykaldi.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-x86_64/egg
running install_lib
running build_py
running build_ext
Using PYCLIF: /usr/local/bin/pyclif
Using CLIF_MATCHER: /usr/clang/bin/clif-matcher
-- Configuring done
-- Generating done
-- Build files have been written to: /home/soroushh/KhodnevisProjects/wav2vec-u/pykaldi/build
[6/505] Building CXX object kaldi/matrix/CMakeFiles/_matrix_ext.dir/matrix-ext.cc.o
FAILED: kaldi/matrix/CMakeFiles/_matrix_ext.dir/matrix-ext.cc.o
/usr/bin/c++ -D_matrix_ext_EXPORTS -I../kaldi/lib -I../kaldi -Ikaldi -I../tools/kaldi/src -I/usr/include/python2.7 -I/home/soroushh/.local/lib/python3.7/site-packages/numpy/core/include -std=c++11 -I.. -isystem /home/soroushh/KhodnevisProjects/wav2vec-u/pykaldi/tools/kaldi/tools/openfst-1.6.7/include -O1 -Wall -Wno-sign-compare -Wno-unused-local-typedefs -Wno-deprecated-declarations -Winit-self -DKALDI_DOUBLEPRECISION=0 -DHAVE_EXECINFO_H=1 -DHAVE_CXXABI_H -DHAVE_MKL -I/opt/intel/mkl/include -m64 -msse -msse2 -pthread -g -fPIC -Wno-maybe-uninitialized -fPIC -MD -MT kaldi/matrix/CMakeFiles/_matrix_ext.dir/matrix-ext.cc.o -MF kaldi/matrix/CMakeFiles/_matrix_ext.dir/matrix-ext.cc.o.d -o kaldi/matrix/CMakeFiles/_matrix_ext.dir/matrix-ext.cc.o -c ../kaldi/matrix/matrix-ext.cc
In file included from /home/soroushh/.local/lib/python3.7/site-packages/numpy/core/include/numpy/ndarraytypes.h:4:0,
from /home/soroushh/.local/lib/python3.7/site-packages/numpy/core/include/numpy/ndarrayobject.h:12,
from /home/soroushh/.local/lib/python3.7/site-packages/numpy/core/include/numpy/arrayobject.h:4,
from ../kaldi/matrix/matrix-ext.cc:8:
/home/soroushh/.local/lib/python3.7/site-packages/numpy/core/include/numpy/npy_common.h:386:9: error: ‘Py_hash_t’ does not name a type
typedef Py_hash_t npy_hash_t;
^~~~~~~~~
In file included from /home/soroushh/.local/lib/python3.7/site-packages/numpy/core/include/numpy/ndarrayobject.h:12:0,
from /home/soroushh/.local/lib/python3.7/site-packages/numpy/core/include/numpy/arrayobject.h:4,
from ../kaldi/matrix/matrix-ext.cc:8:
/home/soroushh/.local/lib/python3.7/site-packages/numpy/core/include/numpy/ndarraytypes.h:655:9: error: ‘npy_hash_t’ does not name a type; did you mean ‘npy_half’?
npy_hash_t hash;
^~~~~~~~~~
npy_half
[11/505] Building CXX object kaldi/chain/CMakeFiles/_chain_generic_numerator.dir/chain-generic-numerator-clifwrap.cc.o
ninja: build stopped: subcommand failed.
Command '['ninja', '-j', '6']' returned non-zero exit status 1.
I also tried different versions of PyKaldi, i.e. 1.8.0 and 1.10.0. But it shows the same error.
Edit 1: when I execute /usr/bin/ninja -j 6 I get the following error: ninja: error: loading 'build.ninja': No such file or directory
Firstly, I tried
pip install pyaudio
But that didn't work... I found that I need PortAudio, so installed it with brew:
brew install portaudio
That worked and then I restarted my Mac.
But pip install pyaudio still doesn't work and I get the below error message:
ERROR: Command errored out with exit status 1:
command: /Applications/Xcode.app/Contents/Developer/usr/bin/python3 -u -c
'import sys, setuptools, tokenize; sys.argv[0] =
'"'"'/private/var/folders/12/44t6pktj77xbqmyp7x368tdr0000gn/T/pip-install-gvxfbjmp/pyaudio/setup.py'"'"';
file='"'"'/private/var/folders/12/44t6pktj77xbqmyp7x368tdr0000gn/T/pip-install-gvxfbjmp/pyaudio/setup.py'"'"';f=getattr(tokenize,
'"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"',
'"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))'
bdist_wheel -d
/private/var/folders/12/44t6pktj77xbqmyp7x368tdr0000gn/T/pip-wheel-gvmpobvy
cwd: /private/var/folders/12/44t6pktj77xbqmyp7x368tdr0000gn/T/pip-install-gvxfbjmp/pyaudio/
Complete output (16 lines):
running bdist_wheel
running build
running build_py
creating build
creating build/lib.macosx-10.14-x86_64-3.7
copying src/pyaudio.py -> build/lib.macosx-10.14-x86_64-3.7
running build_ext
building '_portaudio' extension
creating build/temp.macosx-10.14-x86_64-3.7
creating build/temp.macosx-10.14-x86_64-3.7/src
xcrun -sdk macosx clang -arch x86_64 -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -iwithsysroot/System/Library/Frameworks/System.framework/PrivateHeaders
-iwithsysroot/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.7/Headers
-DMACOSX=1 -I/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.7/include/python3.7m
-c src/_portaudiomodule.c -o build/temp.macosx-10.14-x86_64-3.7/src/_portaudiomodule.o
src/_portaudiomodule.c:29:10: fatal error: 'portaudio.h' file not found
#include "portaudio.h"
^~~~~~~~~~~~~
1 error generated.
error: command 'xcrun' failed with exit status 1
----------------------------------------
ERROR: Failed building wheel for pyaudio
I also tried pip3 install pyaudio without any luck.
BTW I am just doing this because I want to use the Python Speech Recognition API and I think pyaudio is necessary... is it?
Very likely you need to specify the directory path where the compiler can find the source programs like portaudio.h.
Under assumption that the headers are in /usr/local/include directory, try to run the below command in the Terminal:
pip install --global-option='build_ext' --global-option='-I/usr/local/include' --global-option='-L/usr/local/lib' pyaudio
your problem is that with the Xcode command-line. Reinstall using this
xcode-select --install
after xcode is fixed the command('xcrun') should be able to run so just reuse
pip3 install pyaudio
you may need to upgrade pip
python3 -m pip install --upgrade pip
-- Successfully installed pip-21.2.4
and/or invoke pip3
pip3 install ...
I am running mysql#8.x version on my MacOS, I have installed mysql and mysql-connector-o with brew. Currently brew link with mysql.
Running Django project on my python3 virtual env I am receiving following error -
django.core.exceptions.ImproperlyConfigured: Error loading MySQLdb module: No module named 'MySQLdb'.
Did you install mysqlclient or MySQL-python?
and on installing the mysqlclient using pip, I am reciving following error
clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.13.sdk -I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.13.sdk/System/Library/Frameworks/Tk.framework/Versions/8.5/Headers -Dversion_info=(1,3,13,'final',0) -D__version__=1.3.13 -I/usr/local/Cellar/mysql/8.0.12/include/mysql -I/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/include/python3.6m -c _mysql.c -o build/temp.macosx-10.13-x86_64-3.6/_mysql.o
clang -bundle -undefined dynamic_lookup -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.13.sdk build/temp.macosx-10.13-x86_64-3.6/_mysql.o -L/usr/local/Cellar/mysql/8.0.12/lib -lmysqlclient -lssl -lcrypto -o build/lib.macosx-10.13-x86_64-3.6/_mysql.cpython-36m-darwin.so
ld: library not found for -lssl
clang: error: linker command failed with exit code 1 (use -v to see invocation)
error: command 'clang' failed with exit status 1
----------------------------------------
Command "/Volumes/Samsung_T5/Work/python/repos/venv3/bin/python3.6 -u -c "import setuptools, tokenize;__file__='/private/tmp/pip-install-kf2fkhtc/mysqlclient/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /private/tmp/pip-record-qwpjjw3e/install-record.txt --single-version-externally-managed --compile --install-headers /Volumes/Samsung_T5/Work/python/repos/venv3/bin/../include/site/python3.6/mysqlclient" failed with error code 1 in /private/tmp/pip-install-kf2fkhtc/mysqlclient/
As jordanm pointed it out, the issue was exactly with the missing openssl libraries, for which I followed these steps to fix my issue -
Installing openssl
brew install openssl
Now
pip install mysqlclient should work.
If it doesn't and still shows the same error library not found for -lssl, you could also try to link against brew's openssl:
env LDFLAGS="-I/usr/local/opt/openssl/include -L/usr/local/opt/openssl/lib" pip install mysqlclient
if this still does not work, you might need to use the --no-cache option of pip, e.g.
env LDFLAGS="-I/usr/local/opt/openssl/include -L/usr/local/opt/openssl/lib" pip --no-cache install mysqlclient
After installing openssl via Homebrew I placed
export PATH="/usr/local/opt/openssl/bin:$PATH"
export LDFLAGS="-L/usr/local/opt/openssl/lib"
export CPPFLAGS="-I/usr/local/opt/openssl/include"
in my .bash_profile and ran
source .bash_profile
and then
pip install mysqlclient
I'm trying to install Flask-socketIO using pip :
pip install flask-socketIO
But in the end I'm facing this error :
cc -O2 -fPIC -Wimplicit -DLIBEV_EMBED=1 -DEV_COMMON= -DEV_CLEANUP_ENABLE=0 -DEV_EMBED_ENABLE=0 -DEV_PERIODIC_ENABLE=0 -Ibuild/temp.linux-x86_64-2.7/libev -Ilibev -I/home/abhishek/.virtualenvs/test/include -c gevent/gevent.core.c -o build/temp.linux-x86_64-2.7/gevent/gevent.core.o
gevent/gevent.core.c:9:22: fatal error: pyconfig.h: No such file or directory
#include "pyconfig.h"
^
compilation terminated.
error: command 'cc' failed with exit status 1
----------------------------------------
Command "/home/abhishek/.virtualenvs/test/bin/pypy -c "import setuptools,
Looks like you need to have the Python Development package installed.
On CentOS / RHEL:
yum install python-devel
yum install python-lxml
On Ubuntu / Debian:
sudo apt-get install python-dev libxml2-dev libxslt-dev
Having looked at other similar threads, I still can't get pycrypto running.
I'm trying to get it working on my Ubuntu laptop - but I couldn't manage it on my Windows PC either.
I downloaded pycrypto-2.6, extracted it, and ran
python setup.py build
but then this happened
warning: GMP or MPIR library not found; Not building Crypto.PublicKey._fastmath.
building 'Crypto.Hash._MD2' extension
gcc -pthread -fno-strict-aliasing -fwrapv -Wall -Wstrict-prototypes -fPIC -std=c99 -O3 - fomit-frame-pointer -Isrc/ -I/usr/include/python2.7 -c src/MD2.c -o build/temp.linux-i686-?2.7/src/MD2.o
src/MD2.c:31:20: fatal error: Python.h: No such file or directory
compilation terminated.
error: command 'gcc' failed with exit status 1
Would appreciate any help.
You need to install the Python development files. I believe this will do it:
sudo apt-get install python-dev
On Ubuntu, I needed some other packages for it to succeed:
apt-get install autoconf g++ python2.7-dev
pip install pycrypto
On Ubuntu and if you use Python 3.x you will need:
sudo apt-get install gcc python3-dev
you probably already have gcc but just in case if you are trying this command from Dockerfile with base image python:3.6.4-slim-jessie then you will also need gcc.
August 2021
For python 3.8 users run
sudo apt-get install python3.8-dev
and try to install pycrypto again
pip install pycrypto