I'm trying to install a library pyleargist. It requires another lib libfftw3 to be manually installed which I've installed. Since I don't have the root privilege, I have to install libfftw3 under my home directory: ~/usr/include and ~/usr/lib. Then I follow this post: https://superuser.com/questions/242190/how-to-install-matplotlib-on-os-x, added:
export LDFLAGS="-L~/usr/lib"
export CFLAGS="-I~/usr/include
So that pip knows it have to consult /usr to get the include (.h files) and lib (.a, *.so files). However, while running pip install --user pyleargist, it complains about:
gcc-4.4.real: src/leargist.c: No such file or directory
gcc-4.4.real: no input files
error: command 'gcc' failed with exit status 1
I guess what happened is that the path is incorrect so that it can't find the *.c files (I think pip should have downloaded the file somewhere but not sure where it is).
So my questions are the following: 1) in this particular case, how can I install pyleargist with include and lib path under ~/usr? 2) more generally, how can one provide additional path for pip so that it knows where to get the additional include files or libs if not found in the default path?
p.s I am on an ubuntu machine without sudo privilege.
ref:
https://pypi.python.org/pypi/pyleargist/1.0.1
http://www.fftw.org/
pip has a --global-option flag
You can use it to pass additional flags to build_ext.
For instance, to add a -I flag:
pip install --global-option=build_ext --global-option="-I/home/users/abc/include/" pyOpenSSL
This was a helpful thread. Just to add on to this, you can also use pip without root if you pass the --user flag at the end:
pip install --global-option="-I/home/users/abc/include/" mpi4py --user
For example, if you're using python-v2.7, the above command installs the python package to /home/username/.local/lib/python2.7/site-packages
if you dont have root you can get a virtual enviroment no root is needed to get one and your path will be in home
curl -O https://pypi.python.org/packages/source/v/virtualenv/virtualenv-1.10.1.tar.gz
tar xvfz virtualenv-1.10.1.tar.gz
cd virtualenv-1.10.1.tar.gz
python virtualenv.py myVE
then your path is set in your home:
cd myVE/bin
./python
>>> import sys
>>> sys.path
['', '/home/foobar/temp/virtualenv-1.10.1/myVE/lib/python33.zip', '/home/foobar/temp/virtualenv-1.10.1/myVE/lib/python3.3', '/home/foobar/temp/virtualenv-1.10.1/myVE/lib/python3.3/plat-linux', '/home/foobar/temp/virtualenv-1.10.1/myVE/lib/python3.3/lib-dynload', '/usr/lib64/python3.3', '/usr/lib/python3.3', '/usr/lib/python3.3/plat-linux', '/home/foobar/temp/virtualenv-1.10.1/myVE/lib/python3.3/site-packages']
>>>
Related
Issue:
#python3 setup.py install
is failing
Environment: RHEL 8 UBI container
I have a setup.py that looks like this: (any pointers or updates on this setup.py would be appreciated)
import setuptools
print('python/setup.py')
setuptools.setup(
name="process_data",
version="1.0",
description="desc",
author="FirstName LastName",
author_email="dude#abides.abide",
url="https://some.com",
packages=setuptools.find_packages(),
)
I have a directory structure like so (<some>.py in every directory, along with __init__.py):
python/
setup.py
<dir>/__init__.py
<dir>/__init__.py
<dir4>/__init__.py
<dir4>/<sub-dir1>/__init__.py
<dir4>/<sub-dir1>/<sub-dir>__init__.py
I want code in each of these directories to be part of a module package so I can import, nested however deep.
I'm doing this:
#cd python
#python3 setup.py install
I get this error:
python/setup.py
running install
error: can't create or remove files in install directory
The following error occurred while trying to add or remove files in the
installation directory:
/usr/local/lib/python3.6/site-packages/test-easy-install-1625.write-test
[Errno 2] No such file or directory: '/usr/local/lib/python3.6/site-packages/test-easy-install-1625.write-test'
The installation directory you specified (via --install-dir, --prefix, or
the distutils default setting) was:
/usr/local/lib/python3.6/site-packages/
This directory does not currently exist. Please create it and try again, or
choose a different installation directory (using the -d or --install-dir
option).
Questions:
Where is this coming from: test-easy-install-1625.write-test ? (so confused as to what this is)
On my system, indeed this directory does not exist (why is python3 setup.py install looking there? what told it to look there?):
/usr/local/lib/python3.6
There is nothing in my setup.py to indicate to look there.
I looked all over, and it looks like my /site-packages/ directory is here (where other modules are installed, when I ran pip)
`/usr/local/lib64/python3.6/site-packages/`
How do I fix this?
Thank you, I so badly need to get this working right away. I have been working on Windows, and >python setup.py install simply works there.
Are there any env variables I should have setup?
I simply installed python with: dnf -y install python36 that's it. (the latest version available in any RHEL 8 package repo for installation).
I ran into this same error and I was able to fix it by creating a symbolic link. I accept someone else maybe able to offer a more elegant solution and I welcome it.
sudo mkdir /usr/local/lib/python3.6
sudo ln -s /usr/lib/python3.6/site-packages /usr/local/lib/python3.6/
I assume you had enter python command mode, if you don't, the following commands are needed, notice to set variable Path and PythonPath of yours:
setlocal
set env_name=python3.6
set Path=C:\Users\s41167\Miniconda3\envs\%env_name%;C:\Users\s41167\Miniconda3\envs\%env_name%\Scripts;%Path%
set PythonPath=C:\Users\s41167\Miniconda3\envs\%env_name%\Lib\site-packages
cd C:\Users\s41167\Documents\%env_name%
cmd.exe /K activate %env_name%
if you enter python command mode, you can use pip to insall the package:
pip install <package location> --target <install location> --upgrade
below is my install command and result:
(python3.6) C:\Users\s41167\Documents\python3.6>pip install ./python_package --t
arget ./target --upgrade
Processing c:\users\s41167\documents\python3.6\python_package
Building wheels for collected packages: process-data
Building wheel for process-data (setup.py) ... done
Created wheel for process-data: filename=process_data-1.0-cp36-none-any.whl si
ze=1721 sha256=ef5eaf061000f30d472e5dde268694733e40a3f8a4a29fa78faec69f125443c9
Stored in directory: C:\Users\s41167\AppData\Local\Temp\pip-ephem-wheel-cache-
v6lt_ndp\wheels\61\01\f5\07e0760baa10d63e1c43b37eadbb55b79828f4fe337209026a
Successfully built process-data
Installing collected packages: process-data
Successfully installed process-data-1.0
(python3.6) C:\Users\s41167\Documents\python3.6>
and the file arrangement:
.\python_package
__init__.py
setup.py
\directory
__init__.py
\directory4
__init__.py
\sub_dirctory1
__init__.py
in Linux use
sudo python3 setup.py install
in windows use
python setup.py install
or
python3 setup.py install
I don't understand this...
I want to install this https://gist.github.com/sixtenbe/1178136.
It is a peak detection script for python.
Everywhere I look I am told to use pip with the .git extension.
All I see is how to download the .zip, but from there I am lost.
How can I install this?
Thanks.
You can get the individual files in the Gist (or download the Gist as an ZIP and extract) and put them in your source code folder.
Then you will be able to import them as modules in your own scripts:
import analytic_wfm as AW
AW.ACV_A6( ... )
import peakdetect as PK
PK.peakdetect_parabola( ... )
Let's give it another look.
By "installing a package" we might mean that the package should be available via import.
For that the package directory should reside either in the current directory or in one of the other directories in the import search path.
One such directory is the "user-specific site-packages directory, USER_SITE":
python -c "import site; print(site.getusersitepackages())"
Git URL
First we might need a Git URL. Going to https://gist.github.com/sixtenbe/1178136 we can click on the Embed pop-up and switch it to Clone via HTTPS:
in order to obtain the GIT URL: https://gist.github.com/1178136.git.
git and bash
Having the Git URL and the Unix shell (bash) we can install the package manually into the USER_SITE.
Let's go into the USER_SITE first:
cd $(python -c "import site; print(site.getusersitepackages())")
pwd
Now that we are in the USER_SITE, let's download the Gist:
git clone https://gist.github.com/1178136.git analytic_wfm
Finally, let's verify that the package is now available:
cd && python -c "import analytic_wfm.analytic_wfm; print(analytic_wfm.analytic_wfm.__all__)"
If numpy is installed, it prints
['ACV_A1', 'ACV_A2', 'ACV_A3', 'ACV_A4', 'ACV_A5', 'ACV_A6', 'ACV_A7', 'ACV_A8']
pip
Let's try to install a Gist package with pip.
For pip install we should prefix the Git URL with git+:
pip install --user git+https://gist.github.com/1178136.git
This gives us the error:
ERROR: git+https://gist.github.com/1178136.git does not appear to be a
Python project: neither 'setup.py' nor 'pyproject.toml' found.
Looks like the package we've picked is missing the necessary pip configuration!
Let's try another one:
pip install --user git+https://gist.github.com/bf91613a021a536c7ce16cdba9168604.git
Installs NP:
Successfully built llog
Installing collected packages: llog
Successfully installed llog-1.0
Particularly because it has the setup.py.
Note also that Gist does not support subfolders, and pip seems to depend on them in handling the packages argument, but the code in setup.py can workaround this by creating the package subfolder on the fly and copying the Python files there!
Hence if you want to import that Gist, https://gist.github.com/sixtenbe/1178136, with the rest of the requirements.txt dependencies, - you can fork it and add setup.py to the effect.
pypi
Given that the analytic-wfm can also be found at the Python Package Index, https://pypi.org/project/analytic-wfm/, you can install it with
pip install analytic-wfm
I am using pip and trying to install a python module called pyodbc which has some dependencies on non-python libraries like unixodbc-dev, unixodbc-bin, unixodbc. I cannot install these dependencies system wide at the moment, as I am only playing, so I have installed them in a non-standard location. How do I tell pip where to look for these dependencies ? More exactly, how do I pass information through pip of include dirs (gcc -I) and library dirs (gcc -L -l) to be used when building the pyodbc extension ?
pip has a --global-option flag
You can use it to pass additional flags to build_ext.
For instance, to add a --library-dirs (-L) flag:
pip install --global-option=build_ext --global-option="-L/path/to/local" pyodbc
gcc supports also environment variables:
http://gcc.gnu.org/onlinedocs/gcc/Environment-Variables.html
I couldn't find any build_ext documentation, so here is the command line help
Options for 'build_ext' command:
--build-lib (-b) directory for compiled extension modules
--build-temp (-t) directory for temporary files (build by-products)
--plat-name (-p) platform name to cross-compile for, if supported
(default: linux-x86_64)
--inplace (-i) ignore build-lib and put compiled extensions into the
source directory alongside your pure Python modules
--include-dirs (-I) list of directories to search for header files
(separated by ':')
--define (-D) C preprocessor macros to define
--undef (-U) C preprocessor macros to undefine
--libraries (-l) external C libraries to link with
--library-dirs (-L) directories to search for external C libraries
(separated by ':')
--rpath (-R) directories to search for shared C libraries at runtime
--link-objects (-O) extra explicit link objects to include in the link
--debug (-g) compile/link with debugging information
--force (-f) forcibly build everything (ignore file timestamps)
--compiler (-c) specify the compiler type
--swig-cpp make SWIG create C++ files (default is C)
--swig-opts list of SWIG command line options
--swig path to the SWIG executable
--user add user include, library and rpath
--help-compiler list available compilers
Building on Thorfin's answer and assuming that your desired include and library locations are in /usr/local, you can pass both in like so:
sudo pip install --global-option=build_ext --global-option="-I/usr/local/include/" --global-option="-L/usr/local/lib" <you package name>
Another way to indicate the location of include files and libraries are set relevant environment variables before running pip e.g.
export LDFLAGS=-L/usr/local/opt/openssl/lib
export CPPFLAGS=-I/usr/local/opt/openssl/include
pip install cryptography
Just FYI... If you are having trouble installing a package with pip, then you can use the
--no-clean option to see what is exactly going on (that is, why the build did not work). For instance, if numpy is not installing properly, you could try
pip install --no-clean numpy
then look at the Temporary folder to see how far the build got. On a Windows machine, this should be located at something like:
C:\Users\Bob\AppData\Local\Temp\pip_build_Bob\numpy
Just to be clear, the --no-clean option tries to install the package, but does not clean up after itself, letting you see what pip was trying to do.
Otherwise, if you just want to download the source code, then I would use the -d flag. For instance, to download the Numpy source code .tar file to the current directory, use:
pip install -d %cd% numpy
I was also helped by Thorfin's answer; I was building GTK3+ on windows and installing pygobject, I was having difficulties on how to include multiple folders with pip install.
I tried creating pip config file as per pip documentation. but failed.
the one working is with the command line:
pip install --global-option=build_ext --global-option="-IlistOfDirectories"
# and/or with: --global-option="-LlistofDirectories"
the separator that works with multiple folders in windows is ';' semicolon, NOT colon ':' it might be different in other OS.
sample working command line:
pip install --global-option=build_ext --global-option="-Ic:/gtk-build/gtk/x64/release/include;d:/gtk-build/gtk/x64/release/include/gobject-introspection-1.0" --global-option="-Lc:\gtk-build\gtk\x64\release\lib" pygobject==3.27.1
you can use '' or '/' for path, but make sure do not type backslash next to "
this below will fail because there is backslash next to double quote
pip install --global-option=build_ext --global-option="-Ic:\willFail\" --global-option="-Lc:\willFail\" pygobject==3.27.1
Have you ever used virtualenv? It's Python package that let's you create and maintain multiple isolated environments on one machine. Each can use different modules independent of one another without screwing up dependencies in your system library or a separate virtual environment.
If you don't have root privileges, you can download and use the virtualenv package from source:
$ curl -O https://pypi.python.org/packages/source/v/virtualenv/virtualenv-X.X.tar.gz
$ tar xvfz virtualenv-X.X.tar.gz
$ cd virtualenv-X.X
$ python virtualenv.py myVE
I followed the above steps this weekend on Ubuntu Server 12.0.4 and it worked perfectly. Each new virtual environment you create comes with PIP by default so installing packages into your new environment is easy.
Just in case it's of help to somebody, I still could not find a way to do it through pip, so ended up simply downloading the package and doing through its 'setup.py'. Also switched to what seems an easier to install API called 'pymssql'.
I have a git repository with many folders, one of them being a python module installable with pip, like this:
repo.git/
repo.git/folder1/
repo.git/folder2/
repo.git/mymodule/
repo.git/mymodule/__init__.py
repo.git/mymodule/setup.py
repo.git/mymodule/...
Right now I have to do the following to install:
git clone http://server/repo.git
cd repo
pip install mymodule
cd ..
rm -rf repo
Is it possible to install the module directly with pip without explicitly cloning ?
I tried:
pip install git+https://server/repo.git/mymodule/
pip install git+https://server/repo.git:mymodule/
But I get:
IOError: [Errno 2] No such file or directory: '/tmp/pip-88tlLm-build/setup.py'
There is a pull request regarding this feature, and it seems to have been merged to develop branch a month ago. The syntax is the following:
pip install -e git+https://git.repo/some_repo.git#egg=version_subpkg&subdirectory=repo # install a python package from a repo subdirectory
We probably have to wait for a while until it gets merged to master and is distributed.
UPDATE: This is now available and documented at https://pip.pypa.io/en/stable/cli/pip_install/#vcs-support as follows:
For projects where setup.py is not in the root of project,
"subdirectory" component is used. Value of "subdirectory" component
should be a path starting from root of the project to where setup.py
is located.
So if your repository layout is:
- pkg_dir/
- setup.py # setup.py for package ``pkg``
- some_module.py
- other_dir/
- some_file
- some_other_file
You'll need to use
pip install -e vcs+protocol://repo_url/#egg=pkg&subdirectory=pkg_dir
Note: On Windows, you must place the URL in double quotes, or you'll get an error "'subdirectory' is not recognized as an internal or external command". E.g., use:
pip install -e "vcs+protocol://repo_url#egg=pkg&subdirectory=pkg_dir"
It's been already stated in one of the comments under the correct answer, but just to highlight this issue: when executing this from Linux command line, you must escape the &-character since ampersand is telling the command line to run a command in background:
git+https://git.repo/some_repo.git#egg=version_subpkg\&subdirectory=repo
Notice the backslash before the ampersand. The escaping behaviour might depend on the Linux distro; I'm not an expert.
If you ignore this, you might run into a cryptic error like the following:
bash: (...) command not found
I want to install Sphinx 1.1.3 for python 2.6. However, I don't have sudo rights. So instead of installing it in the default place, I want to set a different location, using --prefix. Doing the following:
-bash-3.2$ easy_install Sphinx-1.1.3-py2.6.egg --prefix=/homes/ndeklein/python2.6/site-packages/
gives me:
error: can't create or remove files in install directory
The following error occurred while trying to add or remove files in the
installation directory:
[Errno 13] Permission denied: '/usr/lib/python2.4/site-packages/test-easy-install-18534.write-test'
The installation directory you specified (via --install-dir, --prefix, or
the distutils default setting) was:
/usr/lib/python2.4/site-packages/
Am I typing something wrong with the prefix? Also, what I could use instead (which I've used with other packages):
python setup.py install --home=/homes/ndeklein/python2.6/site-packages/
but I can't find the setup.py script. I'm guessing that EGGs don't have a setup.py script, is that true?
You need to specify options before the package, so the command should be:
easy_install --prefix=/homes/ndeklein/python2.6/site-packages/ Sphinx-1.1.3-py2.6.egg
This website discusses non-root python installs. It might be useful to you...
http://www.astropython.org/tutorials/user-rootsudo-free-installation-of-python-modules7/
To quote a little bit of it:
A user configuration file, ~/.pydistutils.cfg, will override the internal system path for python package installations, redirecting the built libraries (lib), scripts (bin) and data (share) into user owned and specified directories. You must simply tell the python installer where theses directories are located.
The user file, ~/.pydistutils.cfg, has the following lines, using a pretty obvious syntax:
[install]
install_scripts = ~/usr/bin
install_data = ~/usr/share
install_lib = ~/usr/lib/python2.4/site-packages
Of course, whatever directories you specify there should probably exist and you should put them at the front of your PYTHONPATH:
export PYTHONPATH=~/usr/lib/python2.4/site-packages:${PYTHONPATH}
It also looks like more modern python installations (compared to the things in the link) should be able to use the ~/.local directory:
easy_install --prefix=~/.local ...
There is also:
easy_install --user ...
which will install to a user-specific site directory.
You could try using pip install of easy_install(pip is recommended over easy_install these days)
Then you can just use
pip install --user Sphinx
see http://www.pip-installer.org/en/latest/installing.html on how to install pip if needed
You may also want to pip install virtualenv and work inside virtualenv(where pip will install all packages in a local site packages folder). see http://pypi.python.org/pypi/virtualenv for more info.