Develop Reference

Ubuntu 20.04 upgrade, Python missing libffi.so.6

I recently upgraded my OS to Ubuntu 20.04 LTS.
Now when I try to import a library like Numpy in Python, I get the following error:
ImportError: libffi.so.6: cannot open shared object file: No such file or directory
I tried installing the libffi package, but apt can't locate it :
sudo apt-get install libffi
Reading package lists... Done
Building dependency tree
Reading state information... Done
E: Unable to locate package libffi

It seems like I fixed it. I could be wrong, but here is what I think happened:
Ubuntu 20.04 upgraded libffi6 to libffi7
Python is still looking for libffi6
What I did to fix it :
Locate libffi.so.7 in your system
$ find /usr/lib -name "libffi.so*"
Create a simlink named libffi.so.6 that points to libffi.so.7:
sudo ln -s /usr/path/to/libffi.so.7 /usr/lib/path/to/libffi.so.6
UPDATE:
As noted by many users, this fix could have unintended consequences. The better way to do it is to reinstall python as #amichaud explained. This should be used as a last resort IF you're not using pyenv/virtualenv/etc in which case removing python will cause a lot of dependencies to be removed as well.

If you are using pyenv, you should just uninstall the used python version and then reinstall it.
Example:
pyenv uninstall 3.7.4
pyenv install 3.7.4
It's cleaner and safer than renaming system library from my point of view.

I am using Xubuntu 20.04 and recompiling the python version 3.7 did not work for me.
The way I solved this was to download the 19.10 version of the package from here:
http://mirrors.kernel.org/ubuntu/pool/main/libf/libffi/libffi6_3.2.1-8_amd64.deb
and then installing it
sudo apt install ./libffi6_3.2.1-8_amd64.deb
This will unpack the libffi.so.6 and libffi.so.6.0.4 files to /usr/lib/x86_64-linux-gnu/. The libffi.so.6 file is just a link to libffi.so.6.0.4 in the same directory.
As far as I could see this does not overwrite any files so should be safe.
Ubuntu 22.04 additional step
As per comment from pijing below, you need to run this command after installing the above:
apt install libffi-devel
Then recompile Python.

Ubuntu 20 has libffi7 installed instead. It's possible to install the previous version using coming from Ubuntu 19.10 (Eoan Ermine) download from here Or you can follow these commands
$ curl -LO http://archive.ubuntu.com/ubuntu/pool/main/libf/libffi/libffi6_3.2.1-8_amd64.deb
$ sudo dpkg -i libffi6_3.2.1-8_amd64.deb

Same problem for me
Upgraded to Ubuntu 20
pip didn't work anymore (same error)
What I did was:
Delete the virtual env I was using
Recreate it
Sure, I wasn't able to do a pip freeze to get save my dependencies (as pip didn't work), but fortunately I didn't care about them.

The libffi6 package may be downloaded and installed as follows:
Identify a source for apt from the list Download Page for libffi6
(I picked http://mirrors.kernel.org/ubuntu/ for instance)
Make a back up of /etc/apt/sources.list (just in case)
Edit /etc/apt/sources.list and add the line (I added it to the very end of the file) ands save the file
deb https://mirrors.kernel.org/ubuntu bionic main
Update to use the new repository
sudo apt update
Finally, install the package:
sudo apt install libffi6
Note that both libffi6 and libffi7 appear to coexist. (My Ubuntu version is 20.04)
$ sudo apt list | grep libffi[67]/
WARNING: apt does not have a stable CLI interface. Use with caution in scripts.
libffi6/bionic,now 3.2.1-8 amd64 [installed]
libffi7/focal,now 3.3-4 amd64 [installed,automatic]

I had the same problem (when I upgrded to Ubuntu 20.04) when I tried to run Jupyter Notebook.
Step 1) --> Just re-install python3.7.6 (the version I was using) :
$ cd path_to_python3.7_folder
Do again de installation process:
$ ./configure --enable-optimizations
$ make
$ sudo make install
Step 2: uninstall jupyter. I used pip3.7 uninstall ...
See: How to uninstall Jupyter note book installed by pip3
Step 3: Re-install jupyter again:
$ pip3.7 install jupyterlab
$ pip3.7 install notebook
Try to run jupyter again. It should work.

Symbolic linking to higher version of existing libffi,(e.g. pointing 6 to 8 or 9) does NOT harm since libffi's interface is almost frozen up to 9 years from now.
Higher SO version such as 6,7 or 8, simply it's there to indicate minimum requirement in case of new feature availability such as ffi_tramp_is_present. libffi itself is totally backward compatible so far as of 2022.
Anyone can confirm this by browsing inc folder in https://github.com/libffi/libffi
So, if you are seeing this issue in Ubuntu 22, please feel free to create a symbolic link to highest version of so available in your distro.

The problem with libffi can also be tackled with making a symlink:
sudo ln -s /usr/path/to/libffi.so.8 /usr/lib/path/to/libffi.so.7
Then you get another error in xorg log concerning wayland. What helped me was to reinstall wayland and lib32-wayland. After that I could boot normally (although I use xorg in Cinnamon, but also have Gnome installed)

How to install COCO PythonAPI in python3

It seems the COCO PythonAPI only support python2. But peoples do use it in python3 environment.
I tried possible methods to install it, like
python3 setup.py build_ext --inplace
python3 setup.py install
But python3 setup.py install will fail due to coco.py and cocoeval.py containning python2 print function.
Update: solved by updating the COCO PythonAPI project. Leave this question for people facing the same issue.

Try the following steps:
Use git clone to clone the folder into your drive. In this case, it should be git clone https://github.com/cocodataset/cocoapi.git
Use terminal to enter the directory, or open a terminal inside the directory
Type in 2to3 . -w. Note that you might have to install a package to get 2to3. It is an elegant tool to convert code from Python2 to Python3; this code converts all .py files from Python2-compatible to Python3-compatible
Use terminal to navigate to the setup folder
Type in python3 setup.py install
This should help you install COCO or any package intended for Python2, and run the package using Python3. Cheers!

I have completed it with a simple step
pip install "git+https://github.com/philferriere/cocoapi.git#egg=pycocotools&subdirectory=PythonAPI"
** before that you need to install Visual C++ 2015 build tools on your path

Install
Instead of the official version (which has issues with python 3) use an alternative one. Install it on your local machine, globally (i.e., outside any virtual environment). You can do this by:
pip install git+https://github.com/philferriere/cocoapi.git#subdirectory=PythonAPI
Check if it is installed globally:
pip freeze | grep "pycocotools"
You should see something like pycocotools==2.0.0 in your output.
Now, inside your virtual-env (conda or whatever), first install numpy and cython (and maybe setuptools if it's not installed) using pip, and then:
pip install pycocotools
Verify
Inside your project, import (for example) from pycocotools import mask as mask and then print(mask.__author__). This should print out the author's name, which is tsungyi.
Where Is It?
The installed package, like any other packages that are locally installed inside a virtual-env using pip, will go to External Libraries of your project, under site-packages. That means it is now part of your virtual-env and not part of your project. So, other users who may want to use your code, must repeat this installation on their virtual-env as well.
Troubleshooting:
The main source of confusion is that either you did not install the required packages before installing cocoapi, or you did install the required packages but for a different python version. And when you want to check if something is installed, you may check with, for instance, python3.6 and see that it exists, but you are actually running all your commands with python3.7. So suppose you are using python3.7. You need to make sure that:
python -V gives you python3.7 and NOT other version, and pip -V gives you pip 19.2.3 from /home/<USER>/.local/lib/python3.7/site-packages/pip (python3.7), that actually matches with your default python version. If this is not the case, you can change your default python using sudo update-alternatives --config python, and following the one-step instruction.
All the required packages are installed using the right python or pip version. You can check this using pip and pip3 to stop any differences that may cause an issue:
pip freeze | grep "<SUBSTRING-NAME-OF-PACKAGE>" or pip show <PACKAGE-NAME> for more recent versions of pip.
To install the required packages, after you made sure about (1), you need to run:
sudo apt install python-setuptools python3.7-dev python3-wheel build-essential and pip install numpy cython matplotlib
Environment:
The above steps were tested on Ubuntu 18.4, python 3.6.8, pip 19.0.3.

If you are struggling building pycocotools on Ubuntu 20.04 and python3.7
try this:
sudo apt-get install -y python3.7-dev
python3.7 -m pip install pycocotools>=2.0.1

There are alternative versions of the cocoapi that you can download and use too (I'm using python 3.5). Here's a solution that you might want to try out: How to download and use object detection datasets (e.g. coco or pascal)

here's how i did successfully! (the reason is the gcc version)
install the dependencies: cython (pip install cython), opencv (pip install opencv-python)
check the gcc version by this command: gcc --version
your output will be like this 'Command 'gcc' not found, but can be installed with:
sudo apt install gcc
'
Type the below commands to install the gcc:
sudo apt update
sudo apt install build-essential
sudo apt-get install manpages-dev
now check again the gcc version(step2)
if you get below output
'gcc (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0
Copyright (C) 2019 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.'
now run the code for pycocotools installations:
pip install "git+https://github.com/philferriere/cocoapi.git#egg=pycocotools&subdirectory=PythonAPI"
finally wait check if the installation is successful :
'Successfully installed pycocotools-2.0'

Keep getting the "Command "python setup.py egg_info" failed with error code 1" [duplicate]

I'm trying to install some packages with pip.
But pip install unroll gives me
Command "python setup.py egg_info" failed with error code 1 in
C:\Users\MARKAN~1\AppData\Local\Temp\pip-build-wa7uco0k\unroll\
How can I solve this?

About the error code
According to the Python documentation:
This module makes available standard errno system symbols. The value of each symbol is the corresponding integer value. The names and descriptions are borrowed from linux/include/errno.h, which should be pretty all-inclusive.
Error code 1 is defined in errno.h and means Operation not permitted.
About your error
Your setuptools do not appear to be installed. Just follow the Installation Instructions from the PyPI website.
If it's already installed, try
pip install --upgrade setuptools
If it's already up to date, check that the module ez_setup is not missing. If it is, then
pip install ez_setup
Then try again
pip install unroll
If it's still not working, maybe pip didn't install/upgrade setup_tools properly so you might want to try
easy_install -U setuptools
And again
pip install unroll

Here's a little guide explaining a little bit how I usually install new packages on Python + Windows. It seems you're using Windows paths, so this answer will stick to that particular SO:
I never use a system-wide Python installation. I only use virtualenvs, and usually I try to have the latest version of 2.x & 3.x.
My first attempt is always doing pip install package_i_want in some of my Visual Studio command prompts. What Visual Studio command prompt? Well, ideally the Visual Studio which matches the one which was used to build Python. For instance, let's say your Python installation says Python 2.7.11 (v2.7.11:6d1b6a68f775, Dec 5 2015, 20:40:30) [MSC v.1500 64 bit (AMD64)] on win32. The version of Visual Studio used to compile Python can be found here, so v1500 means I'd be using vs2008 x64 command prompt
If the previous step failed for some reason I just try using easy_install package_i_want
If the previous step failed for some reason I go to gohlke website and I check whether my package is available over there. If it's so, I'm lucky, I just download it into my virtualenv and then I just go to that location using a command prompt and I do pip install package_i_want.whl
If the previous step didn't succeed I'll just try to build the wheel myself and once it's generated I'll try to install it with pip install package_i_want.whl
Now, if we focus in your specific problem, where you're having a hard time installing the unroll package. It seems the fastest way to install it is doing something like this:
git clone https://github.com/Zulko/unroll
cd unroll && python setup.py bdist_wheel
Copy the generated unroll-0.1.0-py2-none-any.whl file from the created dist folder into your virtualenv.
pip install unroll-0.1.0-py2-none-any.whl
That way it will install without any problems. To check it really works, just login into the Python installation and try import unroll, it shouldn't complain.
One last note: This method works almost 99% of the time, and sometimes you'll find some pip packages which are specific to Unix or Mac OS X, in that case, when that happens I'm afraid the best way to get a Windows version is either posting some issues to the main developers or having some fun by yourself porting to Windows (typically a few hours if you're not lucky) :)

It was resolved after upgrading pip:
python -m pip install --upgrade pip
pip install "package-name"

I got stuck exactly with the same error with psycopg2. It looks like I skipped a few steps while installing Python and related packages.
sudo apt-get install python-dev libpq-dev
Go to your virtual env
pip install psycopg2
(In your case you need to replace psycopg2 with the package you have an issue with.)
It worked seamlessly.

I got this same error while installing mitmproxy using pip3. The below command fixed this:
pip3 install --upgrade setuptools

Download and install the Microsoft Visual C++ Compiler for Python 2.7 from https://www.microsoft.com/en-in/download/details.aspx?id=44266 - this package contains the compiler and set of system headers necessary for producing binary wheels for Python 2.7 packages.
Open a command prompt in elevated mode (run as administrator)
Firstly do pip install ez_setup
Then do pip install unroll (It will start installing numpy, music21, decorator, imageio, tqdm, moviepy, unroll) # Please be patient for music21 installation
Python 2.7.11 64 bit used

Other way:
sudo apt-get install python-psycopg2 python-mysqldb

I had the same issue when installing the "Twisted" library and solved it by running the following command on Ubuntu 16.04 (Xenial Xerus):
sudo apt-get install python-setuptools python-dev build-essential

It's a dependency issue.
I tried running the following commands helped me sorting out the dependencies, in my case the dependency was
grpcio
pip3 install --upgrade pip
python3 -m pip install --upgrade setuptools
pip3 install --no-cache-dir --force-reinstall -Iv grpcio==1.36.1
pip3 install pulsar-client==2.7.0
remember you must have python3 installed in your system.

First try:
pip install unroll
For sure not work :)
Then Try:
pip2 install unroll
Still get error Try:
pip3 install unroll
If pip3 Worked then suggest to change configuration to use pip3 as pip because you will get a lot of issues as the modern now is Python3 = pip3 if you execute a script files.

I had the same problem.
The problem was:
pyparsing 2.2 was already installed and my requirements.txt was trying to install pyparsing 2.0.1 which throw this error
Context: I was using virtualenv, and it seems the 2.2 came from my global OS Python site-packages, but even with --no-site-packages flag (now by default in last virtualenv) the 2.2 was still present. Surely because I installed Python from their website and it added Python libraries to my $PATH.
Maybe a pip install --ignore-installed would have worked.
Solution: as I needed to move forwards, I just removed the pyparsing==2.0.1 from my requirements.txt.

I ran into the same error code when trying to install a Python module with pip.
#Hackndo noted that the documentation indicate a security issue.
Based on that answer, my problem was solved by running the pip install command with sudo prefixed:
sudo pip install python-mpd2

For me this worked
python3 -m pip3 install -U pip
you can also try
python -m pip install -U pip

pip3 install --upgrade setuptools
WARNING: pip is being invoked by an old script wrapper. This will fail in a future version of pip.
Please see https://github.com/pypa/pip/issues/5599 for advice on fixing the underlying issue.
To avoid this problem you can invoke Python with -m pip instead of running pip directly.
Use python3 -m pip "command", eg:
python3 -m pip install --user pyqt5

I tried all of the above with no success. I then updated my Python version from 2.7.10 to 2.7.13, and it resolved the problems that I was experiencing.

That means some packages in pip are old or not correctly installed.
Try checking version and then upgrading pip.Use auto remove if that works.
If the pip command shows an error all the time for any command or it freezes, etc.
The best solution is to uninstall it or remove it completely.
Install a fresh pip and then update and upgrade your system.
I have given a solution to installing pip fresh here - python: can't open file get-pip.py error 2] no such file or directory

next installation helps me:
pip3 install cython

This worked for me:
sudo xcodebuild -license

Upgrading Python to version 3 fixed my problem. Nothing else did.

I downloaded the .whl file from http://www.lfd.uci.edu/~gohlke/pythonlibs/ and then did:
pip install scipy-0.19.1-cp27-cp27m-win32.whl
Note that the version you need to use (win32/win_amd-64) depends on the version of Python and not that of Windows.

I had this problem using virtualenvs (with pipenv) on my new development setup.
I could only solve it by upgrading the psycopg2 version from 2.6.2 to 2.7.3.
More information is at https://github.com/psycopg/psycopg2/issues/594

I faced the same problem with the same error message but on Ubuntu 16.04 LTS (Xenial Xerus) instead:
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-install-w71uo1rg/poster/
I tested all the solutions provided above and none of them worked for me. I read the full TraceBack and found out I had to create the virtual environment with Python version 2.7 instead (the default one uses Python 3.5 instead):
virtualenv --python=/usr/bin/python2.7 my_venv
Once I activated it, I run pip install unirest successfully.

try on linux:
sudo apt install python-pip python-bluez libbluetooth-dev libboost-python-dev libboost-thread-dev libglib2.0-dev bluez bluez-hcidump

Had the same problem on my Win10 PC with different packages and tried everything mentioned so far.
Finally solved it by disabling Comodo Auto-Containment.
Since nobody has mentioned it yet, I hope it helps someone.

I had the same problem and was able to fix by doing the following.
Windows Python needs Visual C++ libraries installed via the SDK to build code, such as via setuptools.extension.Extension or numpy.distutils.core.Extension. For example, building f2py modules in Windows with Python requires Visual C++ SDK as installed above. On Linux and Mac, the C++ libraries are installed with the compiler.
https://www.scivision.co/python-windows-visual-c++-14-required/

Following below command worked for me
[root#sandbox ~]# pip install google-api-python-client==1.6.4

Methods to solve setup.pu egg_info issue when updating setuptools or not other methods doesnot works.
If CONDA version of the library is available to install use conda instead of pip.
Clone the library repo and then try installation by pip install -e . or by python setup.py install

upgrading python's version did the work for me.

I have just encountered the same problem when trying to pip install -e . a new repo. I did not notice that the contents of setup.py haven't been saved properly and I was effectively running the command with an empty setup.py.
Hence you may experience the same error message if the setup.py of the target package is either empty or malformed.

I solved it on Centos 7 by using:
sudo yum install libcurl-devel

Having trouble installing PyAudio for Python3 on Mint

I was following the instructions here and I'm having trouble getting the installation to work. Basically, the first part works fine. I downloaded portaudio, followed the instructions, and it all seemed to work.
However, when I triedpython3 setup.py install, I got an error. The error came from the /src/_portaudiomodule.c file, and it said that "The file Python.h could not be found". I don't really understand what's going on because there was no Python.h file when I extracted the PyAudio archive. I don't know where the Python.h file was supposed to come from.
I'm kind of a noob to unix systems so I could have easily made a mistake somewhere. I've been trying to solve this for hours and I've had no luck so far. Thanks in advance for your help!

To install the latest version of pyaudio using conda:
source activate -your environment name-
pip install pyaudio
You may run into the following error when installing from pip:
src/_portaudiomodule.c:29:23: fatal error: portaudio.h: No such file or directory
#include "portaudio.h"
compilation terminated.
error: command 'gcc' failed with exit status 1
That is because you don't have the PortAudio development package installed. Install it with:
sudo apt-get install portaudio19-dev

You don't need to compile pyaudio. To install PyAudio, run:
$ sudo add-apt-repository universe
$ sudo apt-get install python-pyaudio python3-pyaudio
The first command enables Universe Ubuntu repository.
If you want to compile it e.g., to use the latest version from git; install build dependencies:
$ sudo apt-get build-dep python-pyaudio python3-pyaudio
After that, you could install it from sources using pip:
$ python3 -mpip install pyaudio
Or to install the current version from git:
$ pip install -e git+http://people.csail.mit.edu/hubert/git/pyaudio.git#egg=pyaudio
Run pip commands inside a virtualenv or add --user command-line option, to avoid modifying the global python3 installation (leave it to the package manager).
I've tested it on Ubuntu. Let me know if it fails on Mint.

I have found the work arround for mac.
please refer the below steps to install pyaudio on python 3.5
Follow these steps :
export HOMEBREW_NO_ENV_FILTERING=1
xcode-select --install
brew update
brew upgrade
brew install portaudio
pip install pyaudio

I was able to get it install with anaconda, using this package.
Follow install instructions for linux here, then do:
conda install -c bokeh pyaudio=0.2.7

try to install using the the below command
pip install pyaudio
after that install the required Microsoft Visual C++ 14.0
refer the below image for the same.
and restart the system and run the same command again
pip install pyaudio

Python.h is nothing but a header file. It is used by gcc to build applications. You need to install a package called python-dev. This package includes header files, a static library and development tools for building Python modules, extending the Python interpreter or embedding Python in applications. To install this package, enter:
sudo apt-get install python3-dev

easy_install and pip giving errors when trying to install numpy

I am running Python 2.7.2 on my machine. I am trying to install numpy with easy_install and pip, but none of them are able to do so. So, when I try:
sudo easy_install-2.7 numpy
I get this error:
"The package setup script has attempted to modify files on your system
that are not within the EasyInstall build area, and has been aborted.
This package cannot be safely installed by EasyInstall, and may not
support alternate installation locations even if you run its setup
script by hand. Please inform the package's author and the EasyInstall
maintainers to find out if a fix or workaround is available."
Moreover, when I try with pip:
sudo pip-2.7 install numpy
I get this error:
RuntimeError: Broken toolchain: cannot link a simple C program
Is there any fix available for this?

I was facing the same error while installing the requirements for my django project. This worked for me.
Upgrade your setuptools version via pip install --upgrade setuptools and run the command for installing the packages again.

you need a compiler and development tools, along with header files for Python.
you didn't mention your OS.
on my system (Ubuntu), I can install python-dev and the toolchain dependencies with:
$ sudo apt-get install python-dev
then I can pip install numpy.