Develop Reference

python3 setup.py install is failing (RHEL 8)

Issue:
#python3 setup.py install
is failing
Environment: RHEL 8 UBI container
I have a setup.py that looks like this: (any pointers or updates on this setup.py would be appreciated)
import setuptools
print('python/setup.py')
setuptools.setup(
name="process_data",
version="1.0",
description="desc",
author="FirstName LastName",
author_email="dude#abides.abide",
url="https://some.com",
packages=setuptools.find_packages(),
)
I have a directory structure like so (<some>.py in every directory, along with __init__.py):
python/
setup.py
<dir>/__init__.py
<dir>/__init__.py
<dir4>/__init__.py
<dir4>/<sub-dir1>/__init__.py
<dir4>/<sub-dir1>/<sub-dir>__init__.py
I want code in each of these directories to be part of a module package so I can import, nested however deep.
I'm doing this:
#cd python
#python3 setup.py install
I get this error:
python/setup.py
running install
error: can't create or remove files in install directory
The following error occurred while trying to add or remove files in the
installation directory:
/usr/local/lib/python3.6/site-packages/test-easy-install-1625.write-test
[Errno 2] No such file or directory: '/usr/local/lib/python3.6/site-packages/test-easy-install-1625.write-test'
The installation directory you specified (via --install-dir, --prefix, or
the distutils default setting) was:
/usr/local/lib/python3.6/site-packages/
This directory does not currently exist. Please create it and try again, or
choose a different installation directory (using the -d or --install-dir
option).
Questions:
Where is this coming from: test-easy-install-1625.write-test ? (so confused as to what this is)
On my system, indeed this directory does not exist (why is python3 setup.py install looking there? what told it to look there?):
/usr/local/lib/python3.6
There is nothing in my setup.py to indicate to look there.
I looked all over, and it looks like my /site-packages/ directory is here (where other modules are installed, when I ran pip)
`/usr/local/lib64/python3.6/site-packages/`
How do I fix this?
Thank you, I so badly need to get this working right away. I have been working on Windows, and >python setup.py install simply works there.
Are there any env variables I should have setup?
I simply installed python with: dnf -y install python36 that's it. (the latest version available in any RHEL 8 package repo for installation).

I ran into this same error and I was able to fix it by creating a symbolic link. I accept someone else maybe able to offer a more elegant solution and I welcome it.
sudo mkdir /usr/local/lib/python3.6
sudo ln -s /usr/lib/python3.6/site-packages /usr/local/lib/python3.6/

I assume you had enter python command mode, if you don't, the following commands are needed, notice to set variable Path and PythonPath of yours:
setlocal
set env_name=python3.6
set Path=C:\Users\s41167\Miniconda3\envs\%env_name%;C:\Users\s41167\Miniconda3\envs\%env_name%\Scripts;%Path%
set PythonPath=C:\Users\s41167\Miniconda3\envs\%env_name%\Lib\site-packages
cd C:\Users\s41167\Documents\%env_name%
cmd.exe /K activate %env_name%
if you enter python command mode, you can use pip to insall the package:
pip install <package location> --target <install location> --upgrade
below is my install command and result:
(python3.6) C:\Users\s41167\Documents\python3.6>pip install ./python_package --t
arget ./target --upgrade
Processing c:\users\s41167\documents\python3.6\python_package
Building wheels for collected packages: process-data
Building wheel for process-data (setup.py) ... done
Created wheel for process-data: filename=process_data-1.0-cp36-none-any.whl si
ze=1721 sha256=ef5eaf061000f30d472e5dde268694733e40a3f8a4a29fa78faec69f125443c9
Stored in directory: C:\Users\s41167\AppData\Local\Temp\pip-ephem-wheel-cache-
v6lt_ndp\wheels\61\01\f5\07e0760baa10d63e1c43b37eadbb55b79828f4fe337209026a
Successfully built process-data
Installing collected packages: process-data
Successfully installed process-data-1.0
(python3.6) C:\Users\s41167\Documents\python3.6>
and the file arrangement:
.\python_package
__init__.py
setup.py
\directory
__init__.py
\directory4
__init__.py
\sub_dirctory1
__init__.py

in Linux use
sudo python3 setup.py install
in windows use
python setup.py install
or
python3 setup.py install

Error while installing Jupyter: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output

I tried installing Jupyter through pip and it failed showing this.
The package setup script has attempted to modify files on your system
that are not within the EasyInstall build area, and has been
aborted.
This package cannot be safely installed by EasyInstall, and may not
support alternate installation locations even if you run its setup
script by hand. Please, inform the package's author and the
EasyInstall maintainers to find out if a fix or workaround is
available.

Try to upgrade your packages:
pip install --upgrade setuptools
pip install --upgrade pip
its seems like missmatch in your versions

simply follow this to install jupyter in windows :-
1nd - Go to file manager and find python folder (like:-python,python37,python39 etc...) may be it's in (C:\Python39) or (C:\Users\prade\AppData\Local\Programs\python39) .
2rd - in python folder open Scripts folder (Ex:- C:\......\python39\Scripts) and copy this path.
3rd - run cmd as administrator and past above path using cd (like:- cd C:\.....\python39\Scripts)
4th - now simply use jupyter install command (like:- pip install jupyterlab)

How to install Python XGBoost package in virtualenv on mac

I have been trying to install Python 2.7 XGBoost on my mac. I am running a framework build of python via brew and trying to install into a virtualenv. I have tried the following methods:
Manual build found here: https://github.com/dmlc/xgboost/blob/master/doc/build.md#python-package-installation
This results in this error:
error: Error: setup script specifies an absolute path:
/Users/username/git/xgboost/python-package/xgboost/../../lib/libxgboost.so
setup() arguments must *always* be /-separated paths relative to the
setup.py directory, *never* absolute paths.
I was able to build xgboost, but I cant install the package. The docs discuss needing disutils, but I can't find it anywhere. I tried running on disutils2, but that didn't work.
pip install found here: https://pypi.python.org/pypi/xgboost/
This results in this error:
Command "python setup.py egg_info" failed with error code 1 in /private/var/folders/jj/7r79983d7jvcz_1y6w2n5v2m0000gn/T/pip-build-l4ak5P/xgboost/

Removing "include_package_data=True" from setup.py helps.
See: https://github.com/django-salesforce/django-salesforce/issues/19

In python-package folder,
modify include_package_data=False in setup.py, and python setup.py install;
run: sudo cp ./xgboost/VERSION you_python_lib_location/site-packages/you_xgboost_version.egg/xgboost/
In my computer, you_python_lib_location is /Users/LeonTao/anaconda2/lib/python2.7 and you_xgboost_version.egg is xgboost-0.6-py2.7.egg. So I run:
sudo cp ./xgboost/VERSION /Users/LeonTao/anaconda2/lib/python2.7/site-packages/xgboost-0.6-py2.7.egg/xgboost/

How to find pg_config path

Complete newbie here, trying to set up Django to work with PostgreSQL.
I'm using mac osx 10.6.8. I have also installed PostgreSQL 9.3
When I run pip install psycopg2 in terminal I get the following error
Downloading/unpacking psycopg2
Downloading psycopg2-2.5.2.tar.gz (685kB): 685kB downloaded
Running setup.py (path:/private/var/folders/A9/A99cs6x0FNusPejCVkYNTE+++TI/-Tmp-/pip_build_bengorman/psycopg2/setup.py) egg_info for package psycopg2
Error: pg_config executable not found.
Please add the directory containing pg_config to the PATH
or specify the full executable path with the option:
python setup.py build_ext --pg-config /path/to/pg_config build ...
or with the pg_config option in 'setup.cfg'.
Complete output from command python setup.py egg_info:
running egg_info
creating pip-egg-info/psycopg2.egg-info
writing pip-egg-info/psycopg2.egg-info/PKG-INFO
writing top-level names to pip-egg-info/psycopg2.egg-info/top_level.txt
writing dependency_links to pip-egg-info/psycopg2.egg-info/dependency_links.txt
writing manifest file 'pip-egg-info/psycopg2.egg-info/SOURCES.txt'
warning: manifest_maker: standard file '-c' not found
Error: pg_config executable not found.
Please add the directory containing pg_config to the PATH
or specify the full executable path with the option:
python setup.py build_ext --pg-config /path/to/pg_config build ...
or with the pg_config option in 'setup.cfg'.
I've seen a number of posts on this
how-to-install-psycopg2-with-pip-on-python
pg-config-executable-not-found
but I have no clue how to find the bin folder location containing pg_config. Any tips on finding this path?

I recommend that you try to use Postgres.app. (http://postgresapp.com)
This way you can easily turn Postgres on and off on your Mac.
Once you do, add the path to Postgres to your .profile file by appending the following:
PATH="/Applications/Postgres.app/Contents/Versions/latest/bin:$PATH"
Only after you added Postgres to your path you can try to install psycopg2 either within a virtual environment (using pip) or into your global site packages.

sudo find / -name "pg_config" -print
The answer is /Library/PostgreSQL/9.1/bin/pg_config in my configuration (MAC Maverick)

Installing homebrew
/usr/bin/ruby -e "$(curl -fsSL https://raw.github.com/gist/323731)"
And then installing postgresql
brew install postgresql
gave me this lovely bit of output:
checking for pg_config... yes
ahhh yeahhhhh

Postgres.app was updated recently. Now it stores all the binaries in "Versions" folder
PATH="/Applications/Postgres.app/Contents/Versions/9.4/bin:$PATH"
Where 9.4 – version of PostgreSQL.

Once you install the current PostgreSQL app on the MacOS X 10.11, this is where the pg_config file is /Library/PostgreSQL/9.5/bin/pg_config.
Then on the Terminal:
$ export PG_HOME=/Library/PostgreSQL/9.5
$ export PATH=$PATH:$PG_HOME/bin
This will put the path in the .profile of whatever terminal you are using.
In your environment (assuming you are using virtualenv) you then install psycopg2:
$ pip install psycopg2
You should see if you had downloaded it before:
Collecting psycopg2
Using cached psycopg2-2.6.1.tar.gz
Installing collected packages: psycopg2
Running setup.py install for psycopg2 ... done
Successfully installed psycopg2-2.6.1

To summarize -- PostgreSQL installs its files (including its binary or executable files) in different locations, depending on the version number and the installation method.
Some of the possibilities:
/usr/local/bin/
/Library/PostgreSQL/9.2/bin/
/Applications/Postgres93.app/Contents/MacOS/bin/
/Applications/Postgres.app/Contents/Versions/9.3/bin/
No wonder people get confused!
Also, if your $PATH environment variable includes a path to the directory that includes an executable file (to confirm this, use echo $PATH on the command line) then you can run which pg_config, which psql, etc. to find out where the file is located.

I had exactly the same error, but I installed postgreSQL through brew and re-run the original command and it worked perfectly :
brew install postgresql

This is how to simply get the path of pg_config
$ which pg_config // prints the directory location
/usr/bin/pg_config

Have same issue on mac, you probably need to
brew install postgresql
then you can run
pip install psycopg2
The brew will fix PATH issue for you
this solution works for me at least.

You can find the pg_config directory using its namesake:
$ pg_config --bindir
/usr/lib/postgresql/9.1/bin
$
Tested on Mac and Debian. The only wrinkle is that I can't see how to find the bindir for different versions of postgres installed on the same machine. It's fairly easy to guess though! :-)
Note: I updated my pg_config to 9.5 on Debian with:
sudo apt-get install postgresql-server-dev-9.5

check /Library/PostgreSQL/9.3/bin and you should find pg_config
I.E. /Library/PostgreSQL/<version_num>/
ps: you can do the following if you deem it necessary for your pg needs -
create a .profile in your ~ directory
export PG_HOME=/Library/PostgreSQL/9.3
export PATH=$PATH:$PG_HOME/bin
You can now use psql or postgres commands from the terminal, and install psycopg2 or any other dependency without issues, plus you can always just ls $PG_HOME/bin when you feel like peeking at your pg_dir.

I used :
export PATH=$PATH:/Library/PostgreSQL/9.6/bin
pip install psycopg2

Works for me by installing the first the following pip packages: libpq-dev and postgresql-common

For people looking for the pg_config path for postgresql installed via brew on Apple silicon: /opt/homebrew/Cellar/postgresql/<postgres_version>/bin.
For Intel based Mac system /usr/local/Cellar/postgresql/<postgres_version>/bin (Need to verify this).

For those using macOS Big Sur and have their default shell as the Z shell, you can add the path to the py_config binary in the .zprofile which is equivalent to the .bash_profile as explained in this link https://support.apple.com/en-us/HT208050
I had similar issues when using PyCharm with a virtual environment within which I was unable to change my shell to bash to get things going. Adding the postgres bin path fixed my issue:
MacBook-Pro ~ % cat .zprofile
# Setting PATH for Postgres
PATH="/Applications/Postgres.app/Contents/Versions/latest/bin:$PATH"
export PATH
Then running this worked without any issues:
pip install psycopg2

path of pg_config in my case (MacOS)
/Library/PostgreSQL/13/bin
Execute the following in the terminal:
PATH="/Library/PostgreSQL/13/bin:$PATH"
Then
pip install psycopg2

Mac + virtualenv + pip + postgresql = Error: pg_config executable not found

I was trying to install postgres for a tutorial, but pip gives me error:
pip install psycopg
A snip of error I get:
Error: pg_config executable not found.
Please add the directory containing pg_config to the PATH
or specify the full executable path with the option:
python setup.py build_ext --pg-config /path/to/pg_config build ...
or with the pg_config option in 'setup.cfg'.
Where is pg_config in my virtualenv? How to configure it? I'm using virtualenv because I do not want a system-wide installation of postgres.

On the Mac, if you're using Postgres.app, the pg_config file is in your /Applications/Postgres.app/Contents/Versions/<current_version>/bin directory. That'll need to be added to your system path to fix this error, like this:
export PATH=$PATH:/Applications/Postgres.app/Contents/Versions/<current_version>/bin
So for example, if the current Postgres.app version is 9.5, this export line would be:
export PATH=$PATH:/Applications/Postgres.app/Contents/Versions/9.5/bin
With more recent versions of the Postgres.app (> 9.5?), you can simply add "latest" in place of the version number, like so:
export PATH=$PATH:/Applications/Postgres.app/Contents/Versions/latest/bin

On Mac, the solution is to install postgresql:
brew install postgresql
On CentOS, the solution is to install postgresql-devel:
sudo yum install postgresql-devel
pg_config is in postgresql-devel package

I totally agree with john hight that most of posted answers are totally offtopic assuming the OP exactly specified need of using virtualenv.
For me the answer was runing following command in prompt while having activated virtualenv:
export PATH="/Applications/Postgres.app/Contents/Versions/9.4/bin:$PATH"
(notice that part 9.4 stands for version and may vary)
or if you want to use the latest installed version of Postgres:
export PATH="/Applications/Postgres.app/Contents/Versions/latest/bin:$PATH"
and then:
pip install psycopg2
goes sucesfully assuming you have installed postgres. And if not, then remember that the best and recomended solution is to use: Postgres.app

Don't forget that your $PATH variable in the virtual environment != your global $PATH variable. You can confirm this with 'echo $PATH' in your virtualenv and also in a new shell. So, unless you want to install PostgreSQL as a unique instance inside your virtual environment (not a thing worth doing, imo), you'll need to modify the $PATH variable within the virtualenv to include the path to your global installation (which will solve your missing pg_config error).
Here are the steps:
1.) In a new shell, type 'which pg_config'. This will return the path. Copy it. In my case, the path looked like this: /Applications/Postgres.app/Contents/Versions/9.3/bin
2.) Back in your virtualenv shell, type 'export PATH=/your-path-to-pg_config:$PATH'
3.) Then, still within the virtualenv, 'pip install psycopg2'
If all goes according to plan, this will install psycopg2 within the virtual environment, but the installation will refer to your Global PostgreSQL installation. In my case, this Global installation was installed via Postgres.App, hence the path. I prefer this method of working with psycopg2 as it means I can use the database easily within any virtualenv rather than only within the defined virtual environment.
Hope this helps anyone who arrives here. For Google juice, here's the explicit (and vague) error language returned when you run into this problem:
Command python setup.py egg_info failed with error code 1

Here's how I was able to solve this problem on my Mac (OSX 10.9):
brew update
brew install --force ossp-uuid
brew install postgresql
pip install psycopg
I got a CLANG error when I tried pip install psycopg (an LLVM 5.1 issue), so I had to install psycopg with this command instead:
ARCHFLAGS=-Wno-error=unused-command-line-argument-hard-error-in-future pip install psycopg
It's similar to Mingyu's solution, but there are enough differences that I thought it was worth sharing.

you must configure path postgresql:
export PATH=$PATH:/Library/PostgreSQL/11/bin
after, you must install requirements:
pip3 install -r requirements

For OS X El Capitan (10.11.6) + brew + virtualenv + PostgreSQL 9.5:
After installing PostgreSQL 9.5:
brew install postgresql#9.5
Then, open your terminal and execute:
export PATH=$PATH:/usr/local/opt/postgresql\#9.5/bin/
pip install psycopg2

This error is caused when the build tools can't find the Postgresql libraries.
Often it's required to instruct psycopg2 how to find the pg_config binary, you can:
add the path to pg_config in your shell path (/usr/local/pgsql/bin/)
or edit the setup.cfg file in the psycopg2 source folder and provide
the full path to pg_config on the line that starts with pg_config=
pg_config=/usr/local/pgsql/bin/pg_config
the above is an example, you can do locate pg_config to find out where it resides, or simply type which pg_config and it should tell you the path.
Less often the error comes from not having postgresql installed on your system. If so, download and build postgres, or download a pre-built psycopg2 binary for OS X.

virtualenv is for python packages. I don't think you'll be able to contain postgres inside a virtualenv. The error message you're seeing is presumably because you haven't yet installed postgres. The psycopg2 install script is looking for postgres files (in this case pg_config) and not finding them because it is not installed. postgres can't be installed using pip or virtualenv.

On Windows I installed postgres manually from http://www.enterprisedb.com/products-services-training/pgdownload#windows.
After that the same command works.

In addition to the answers provided by #bkev and #andi, according to the documentation on Postgres.app, you should add the following to your bash_profile on Mac:
export PATH=$PATH:/Applications/Postgres.app/Contents/Versions/latest/bin
Note that, there is no hard-coded version number. I wanted to add this as a comment to above answers, but I don't have enough rep for this.

If you're using postgresql 9.4, the file is located in
/usr/pgsql-9.4/bin/pg_config
The name of the package is
postgresql94-9.4.9-1PGDG.rhel6.x86_64
to add pg_config to your PATH, do the following
PATH=$PATH:/usr/pgsql-9.4/bin/

If you don't have to use the psycopg driver specifically, switch to the pg8000 driver. It's pure Python and less finicky.

On Ubuntu I just needed the postgres dev package:
sudo apt-get install postgresql-server-dev-all

Mine was located in /Library/PostgreSQL/9.4/bin
export PATH=$PATH:/Library/PostgreSQL/9.4/bin

pip install psycopg2-binary
Binary install will work in virtual env, it Will have lib files necessary to work in virtual environment

I added to $PATH postgres, but had similar issue. After made a change in Dockerfile and installed through bin/sh, an install passed. It could be also links in with python alpine version. Not alpine version had not any problem during installation.
# syntax=docker/dockerfile:1
FROM python:3.11-rc-alpine
ENV PYTHONDONTWRITEBYTECODE=1
ENV PYTHONUNBUFFERED=1
WORKDIR /code
COPY requirements.txt /code/
RUN #pip install --upgrade pip
RUN /bin/sh -c pip install -r requirements.txt
RUN #pip install -r requirements.txt
COPY . /code/