I'm trying to install the python confluent-kafka package using pip. I'm attempting this on an aws ec2 instance that is running amazon linux (version Amazon Linux AMI release 2016.09). I'm simply doing:
pip install pip install confluent-kafka
This however produces the following error:
In file included from confluent_kafka/src/confluent_kafka.c:17:0:
confluent_kafka/src/confluent_kafka.h:21:32: fatal error: librdkafka/rdkafka.h: No such file or directory
#include <librdkafka/rdkafka.h>
^
compilation terminated.
error: command 'gcc' failed with exit status 1
To resolve this, I did two things:
1) Followed the instructions on this page and added the file /etc/yum.repos.d/confluent.repo with the contents:
[Confluent.dist]
name=Confluent repository (dist)
baseurl=http://packages.confluent.io/rpm/3.0/6
gpgcheck=1
gpgkey=http://packages.confluent.io/rpm/3.0/archive.key
enabled=1
[Confluent]
name=Confluent repository
baseurl=http://packages.confluent.io/rpm/3.0
gpgcheck=1
gpgkey=http://packages.confluent.io/rpm/3.0/archive.key
enabled=1
2) attempted to install the librdkafka library using this command:
sudo yum clean all
sudo yum install -y librdkafka1 librdkafka-devel
Yum spits out this error however:
Error: Package: librdkafka1-0.9.1_confluent3.0.1-1.el7.x86_64 (Confluent.dist)
Requires: openssl-libs
Error: Package: librdkafka1-0.9.1_confluent3.0.1-1.el7.x86_64 (Confluent.dist)
Requires: libsasl2.so.3()(64bit)
After some googling, I tried:
sudo ln /usr/lib64/libsasl2.so.2 /usr/lib64/libsasl2.so.3
This had no effect. I tried doing a yum upgrade and this also did not solve the issue. After much googling, this kafka user group post is the only remotely helpful thing I can find, but it sadly does not contain a solution to the problem.
I'd really like to get kafka python up and running on this instance, so any suggestions would be greatly appreciated.
Amazon Linux 2016.19 seems to be based on RHEL 6 so you will need to reference Confluent's RHEL 6 Yum repo as described in Confluent docs, namely by writing the following to /etc/yum.repos.d/confluent:
[Confluent.dist]
name=Confluent repository (dist)
baseurl=http://packages.confluent.io/rpm/3.1/6
gpgcheck=1
gpgkey=http://packages.confluent.io/rpm/3.1/archive.key
enabled=1
[Confluent]
name=Confluent repository
baseurl=http://packages.confluent.io/rpm/3.1
gpgcheck=1
gpgkey=http://packages.confluent.io/rpm/3.1/archive.key
enabled=1
Followed by:
$ sudo yum clean all
$ sudo yum install gcc librdkafka1 librdkafka-devel cyrus-sasl-devel openssl-libs python-devel
$ pip install confluent-kafka (possibly in a virtualenv)
Related
I've been trying to install Pybossa and I've been trying to follow these steps from the documentation, and this link too (which they're the same).
when it comes to using this command pip install -r requirements.txt it keeps running some installations and then and error occurred :
ERROR: Could not find a version that satisfies the requirement jeepney==0.4 (from pybossa==3.1.2->-r requirements.t
xt (line 3)) (from versions: none)
ERROR: No matching distribution found for jeepney==0.4 (from pybossa==3.1.2->-r requirements.txt (line 3))
this is what's inside requirements.txt :
I don't know why this is happening as i'm following the documentation .. so i'd glade if anyone helped me.
I'm using :
Ubuntu server 18.04
python 2.7
virtualenv
PostgreSQL
You are better off going for latest python 3. python 2 had its eol early this year. For new approaches it would be a dead end.
I am on 16.04, but hope you can do the turn likewise
make use of native venv option, don't use virtualenv any more
during the setup you likely will get compilation errors. Those are due to the fact not having a fully equipped build machine. It's very likely that you need to apt-get install more or less dev packages to your system. It depends how deep you had done gcc compilation in the past. For me it was:
sudo apt-get install python3-dev
sudo apt-get install libsasl2-dev libldap2-dev libssl-devm
whenever you encounter compilation errors like this:
Modules/constants.h:7:18: fatal error: lber.h: No such file or directory
compilation terminated.
you need to find the appropriate dev lib and add it to the system with apt-get like mentioned above by easiest looking the error up in your favorite internet search engine
Steps for the setup of pybossa
python3 -m venv ./pb_env
. ./pb_env/bin/activate
pip install --upgrade pip
mkdir pb #shorthand for pybossa
cd pb
git clone --recursive https://github.com/Scifabric/pybossa
cd pybossa/
pip install -r requirements.txt
feel free to ask again if you still get stuck.
All versions of jeepney starting with 0.1 (released in 2017) requires Python >= 3.5.
I'm trying to run a flask app on EC2, using virtualenv. I've created a virtualenv using virtualenv -p /usr/bin/python venv and activated it using source venv/bin/activate.
Then I cloned my github flask repo, and ran pip install -r requirements.txt. However, I'm getting an error when installing matplotlib.
File "/home/ec2-user/network-visualizer/venv/lib/python2.6/site-
packages/pip/_vendor/cachecontrol/serialize.py", line 81, in dumps
).encode("utf8"),
MemoryError
I've googled around and tried pip --no-cache-dir install matplotlib as suggested here but this time, I'm seeing
================================================================
============
* The following required packages can no
t be built:
* freetype
----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /
tmp/pip-build-3dMFaT/matplotlib
I then ran sudo yum install freetype and got
Package freetype-2.3.11-15.14.amzn1.x86_64 already installed and lat
est version
Nothing to do
I'm stuck now... What should I need to install additionally to make this work?
Try installing the freetype development tools as well. I think that it's freetype-devel or libfreetype.
Try running this
sudo yum install pkg-config
Install the following:
sudo apt install libfreetype6 libfreetype6-dev
That should solve the problem
I recently downloaded Ubuntu 14.04 desktop version alongside Windows 10.
My PC configs are: 4 GB RAM, 64 bit
I installed Anaconda after downloading the file using:
bash Anaconda3-2.5.0-Linux-x86_64.sh
Installation was successful.
Now in my ubuntu terminal, I follwed steps as instructed on http://xgboost.readthedocs.org/en/latest/build.html#building-on-ubuntu-debian
Installed a recent GNU C++ compiler -->successful
git clone --recursive https://github.com/dmlc/xgboost
cd xgboost; make -j4
Building was also successful and I could build 'libxgboost.so'
sudo apt-get install python-setuptools (Successful)
cd python-package
sudo python setup.py install
The 5. command returns an error after a lot of lines:
'ImportError: No module named numpy.distutils.core'
Can anyone suggest how to get rid of this error so that I can install xgboost ?
I had the same problem today. I believe the issue is the instructions you listed are out of date for Python installs, as they are now enabled with pip install.
Delete the xgboost directory that your above install attempt created, and then execute:
pip install xgboost
It should all work with one command. See also the Python Specific XGBoost Install Instructions.
This issue is listed in xgboost's github
The solution is
sudo -s
python setup.py install
instead of
cd python-package; sudo python setup.py install
I was following the instructions here and I'm having trouble getting the installation to work. Basically, the first part works fine. I downloaded portaudio, followed the instructions, and it all seemed to work.
However, when I triedpython3 setup.py install, I got an error. The error came from the /src/_portaudiomodule.c file, and it said that "The file Python.h could not be found". I don't really understand what's going on because there was no Python.h file when I extracted the PyAudio archive. I don't know where the Python.h file was supposed to come from.
I'm kind of a noob to unix systems so I could have easily made a mistake somewhere. I've been trying to solve this for hours and I've had no luck so far. Thanks in advance for your help!
To install the latest version of pyaudio using conda:
source activate -your environment name-
pip install pyaudio
You may run into the following error when installing from pip:
src/_portaudiomodule.c:29:23: fatal error: portaudio.h: No such file or directory
#include "portaudio.h"
compilation terminated.
error: command 'gcc' failed with exit status 1
That is because you don't have the PortAudio development package installed. Install it with:
sudo apt-get install portaudio19-dev
You don't need to compile pyaudio. To install PyAudio, run:
$ sudo add-apt-repository universe
$ sudo apt-get install python-pyaudio python3-pyaudio
The first command enables Universe Ubuntu repository.
If you want to compile it e.g., to use the latest version from git; install build dependencies:
$ sudo apt-get build-dep python-pyaudio python3-pyaudio
After that, you could install it from sources using pip:
$ python3 -mpip install pyaudio
Or to install the current version from git:
$ pip install -e git+http://people.csail.mit.edu/hubert/git/pyaudio.git#egg=pyaudio
Run pip commands inside a virtualenv or add --user command-line option, to avoid modifying the global python3 installation (leave it to the package manager).
I've tested it on Ubuntu. Let me know if it fails on Mint.
I have found the work arround for mac.
please refer the below steps to install pyaudio on python 3.5
Follow these steps :
export HOMEBREW_NO_ENV_FILTERING=1
xcode-select --install
brew update
brew upgrade
brew install portaudio
pip install pyaudio
I was able to get it install with anaconda, using this package.
Follow install instructions for linux here, then do:
conda install -c bokeh pyaudio=0.2.7
try to install using the the below command
pip install pyaudio
after that install the required Microsoft Visual C++ 14.0
refer the below image for the same.
and restart the system and run the same command again
pip install pyaudio
Python.h is nothing but a header file. It is used by gcc to build applications. You need to install a package called python-dev. This package includes header files, a static library and development tools for building Python modules, extending the Python interpreter or embedding Python in applications. To install this package, enter:
sudo apt-get install python3-dev
I am trying to install paver to ubuntu (12.04) with this command
sudo apt-get install python-paver
I got this error message
Reading package lists... Done
Building dependency tree
Reading state information... Done
E: Unable to locate package python-paver
How do I install paver in ubuntu then?
From the documentation:
To install using pip,:
$ pip install -U Paver
To install using easy_install:
$ easy_install -U Paver
I just uploaded a new Debian package to my Launchpad PPA for Ubuntu precise, quantal, raring, saucy, and trusty. I also just uploaded it to the official Debian repos, so it should become part of the official Debian and Ubuntu repos again.